Principal Engineer Capacity/Compute

USD 320,000-485,000 per year
SENIOR
✅ Hybrid
✅ Visa Sponsorship

Tech Stack

API @ 4 Azure @ 7 Data Engineering Data Pipelines DevOps @ 7 GPU @ 3 Grafana @ 4 Kubernetes Machine Learning @ 4 Observability @ 4 Planning @ 4 Profiling @ 4 Prometheus @ 4 Python @ 6 SQL @ 6

Details

Anthropic's Capacity Engineering team builds the production systems used to account for, measure, allocate, and maximize utilization across a large infrastructure fleet spanning multiple accelerator families, CPU families, and cloud providers. The role focuses on data engineering, systems engineering, observability, capacity planning, efficiency, attribution, and forecasting for compute infrastructure.

Responsibilities

  • Build the planning and allocation stack used by leadership, engineering teams, and schedulers, including cross-region and cross-provider placement, guardrails, queueing, and occupancy KPIs.
  • Drive efficiency programs covering stranding and rightsizing, unused capacity recovery, and job-level utilization across training, inference, and evaluation workloads.
  • Establish per-configuration baselines and work with system-owning teams to improve utilization.
  • Own attribution and forecasting by reconciling billing across more than ten providers with telemetry and internal systems, attributing spend to workloads, and converting demand signals and research roadmaps into compute plans and supply pipelines.
  • Build data pipelines that ingest occupancy, utilization, and cost data into BigQuery, with ownership of completeness, latency SLOs, and gap detection.
  • Integrate data from new cloud providers and normalize billing, usage, and telemetry data across heterogeneous environments.
  • Operate Kubernetes-native systems at scale, including collection agents, workload labeling, taint and reservation behavior, and scheduling systems.
  • Define schema contracts, gather requirements, design self-service data products, and support consumers ranging from research engineers to finance and company leadership.
  • Establish on-call processes and SLOs for production systems.
  • Collaborate with research engineering, infrastructure, inference, and finance teams in a high-autonomy, ambiguous environment.

Requirements

  • Strong track record building and operating production systems in a hands-on engineering role with a DevOps focus.
  • Production-quality Python and SQL skills. Pipeline code is primarily Python, while the presentation layer uses BigQuery SQL, including table-valued functions and views.
  • Deep experience with at least one major cloud provider: Amazon Web Services, Google Cloud, or Microsoft Azure.
  • Experience with observability tooling, including Prometheus, PromQL, and Grafana; recording rules and monitoring relied upon by engineering teams.
  • Ability to gather requirements independently and work across organizational boundaries with limited direction.
  • Bachelor's degree or an equivalent combination of education, training, and experience.
  • Relevant field of study demonstrated through coursework, training, or professional experience.

Preferred Qualifications

  • Experience with capacity planning, resource management, or cost attribution systems at a hyperscaler or in a large-scale machine learning environment.
  • Scheduling and packing efficiency experience, or profiling-driven optimization of large distributed workloads.
  • Multi-cloud data ingestion experience, including billing exports, reservation APIs, on-demand capacity reservations, commitments, and vendor telemetry.
  • Total cost of ownership and forecasting experience.
  • Familiarity with accelerator infrastructure, including GPU metrics such as DCGM, TPU utilization, Trainium power and utilization metrics, or machine learning training and inference systems at the hardware level.
  • Experience building internal data products with self-service access, schema contracts, API serving, documentation, and discoverability.
  • Storage efficiency, retention, and lifecycle program experience at exabyte scale.

Benefits

Anthropic offers competitive compensation and benefits, optional equity donation matching, generous vacation and parental leave, flexible working hours, and office space for collaboration.

More jobs at Anthropic

Similar jobs