The company runs one of the largest and fastest-growing infrastructure fleets in the industry, spanning multiple accelerator families, CPU families and cloud providers.
The Capacity Engineering team makes sure every infrastructure resource is accounted for, well utilized and efficiently allocated. It owns the data, tooling and operational systems used to plan, measure and maximize utilization across first-party and third-party compute.
You will build the production systems behind this work: data pipelines that ingest and normalize telemetry from varied cloud environments, observability tooling for real-time fleet health, and instrumentation that measures how efficiently each major workload uses its hardware. You will write production-quality code daily and work with Kubernetes-native infrastructure at significant scale. Your decisions will shape one of the company's largest areas of spend.
You will work closely with research engineering, infrastructure, inference and finance teams. The role suits someone comfortable moving between data engineering, systems engineering and observability, who does well with high autonomy and high ambiguity.
This is a pipeline role feeding four areas. You will focus mainly on one, based on your background and business priorities, but the areas overlap and the boundaries are fluid.
Data platform: pipelines that ingest occupancy and utilization telemetry from Kubernetes clusters, normalize billing and usage across cloud providers, and serve BigQuery tables to the whole organization. Correctness, completeness and latency are central. Consumers range from research engineers to finance and leadership, so the work includes defining schema contracts, making data easy to find, and working out what people need.
Planning: knowing what the fleet has, where it is heading and what blocks it. This covers real-time fleet health tooling, capacity planning platforms, alerts on occupancy drops and allocation problems, and fixes to scheduling and fragmentation. It involves Kubernetes operations on one side and cross-team coordination on the other.
Efficiency: measuring how well each major workload uses its hardware. This means instrumenting utilization across training, inference and eval systems, building benchmarking infrastructure, setting per-configuration baselines, and working directly with the teams that own those systems to close gaps. The metric must be credible enough that the owning team accepts the number.
Attribution and forecasting: linking fleet cost to what the business does with it. This includes reconciling cloud billing exports against vendor telemetry and internal systems with mismatched schemas, assigning spend to the workloads and teams behind it, and turning inference demand and research roadmaps into a defensible compute plan. Efficiency metrics must hold up with finance: free of pure demand and unit-price effects, reproducible month to month, and clear to a CFO.
Responsibilities
Build the planning and allocation stack: the tools leadership uses to allocate capacity, teams use to plan against their allocations, and the scheduler enforces. This covers cross-region and cross-provider placement, guardrails, queueing and occupancy KPIs.
Run efficiency programs covering stranded capacity, rightsizing, recovery of unused capacity, and job-level utilization across training, inference and eval. Set per-configuration baselines and work with system owners to close gaps.
Own attribution and forecasting. Reconcile billing from more than ten providers against telemetry and internal systems, assign spend to the workloads producing it, and turn demand signals and research roadmaps into a defensible compute plan and supply pipeline.
Build the data platform beneath all of this: pipelines that bring occupancy, utilization and cost from a fast-diversifying fleet into BigQuery. You own completeness, latency SLOs and gap detection. Each new provider is a new integration.
Operate Kubernetes-native systems at scale, including collection agents, workload labeling, and the taints, reservations and scheduling behavior that determines which capacity is actually usable.
Treat the output as a product. Gather your own requirements, define schema contracts, and design for consumers from research engineers to a CFO. You also own on-call and SLOs, since these surfaces are critical to the company.
Requirements
A strong record building and running production systems. This is hands-on engineering with a devops emphasis.
Production-quality Python and SQL. Most pipeline code is Python, and the presentation layer is BigQuery SQL, including table-valued functions and views. Both must be idiomatic, well tested and maintainable.
Deep experience with at least one major cloud provider (Amazon Web Services, Google Cloud or Microsoft Azure) and its operations.
Experience with Prometheus, PromQL and Grafana, including writing recording rules and building monitoring that engineering teams depend on.
Ability to gather your own requirements, work across organizational boundaries, and make progress in an ambiguous setting with limited direction.
A bachelor's degree or equivalent combination of education, training and/or experience, in a field relevant to the role as shown through coursework, training or professional experience. Years of experience required follow the internal level for the position.
Preferred: experience with capacity planning, resource management or cost attribution at a hyperscaler or in a large machine learning environment. Product engineering and developer experience count toward this.
Preferred: scheduling and packing efficiency, or profiling-driven optimization of large distributed workloads.
Preferred: multi-cloud data ingestion, especially normalizing billing exports, reservation APIs, on-demand capacity reservations, commitments and vendor telemetry from providers with different billing arrangements.
Preferred: total cost of ownership and forecasting, including determining whether infrastructure growth is caused by or merely correlated with business drivers.
Preferred: accelerator infrastructure familiarity, such as GPU metrics (DCGM), TPU utilization, Trainium power and utilization metrics, or machine learning training and inference systems at the hardware level.
Preferred: building internal data products with self-service access, schema contracts, API serving, documentation and discoverability, with attention to how the data is consumed.
Preferred: storage efficiency, retention and lifecycle programs at exabyte scale.
Conditions
Annual salary range: $320,000 to $485,000 USD.
Location-based hybrid policy: staff are currently expected to be in an office at least 25% of the time. Some roles may require more office time.
Visa sponsorship is available, but not for every role or every candidate. If an offer is made, the company will make every reasonable effort to obtain a visa and retains an immigration lawyer to help.
Candidates are encouraged to apply even if they do not meet every qualification. Equal-opportunity statement: the company seeks a range of diverse perspectives on its team.
Benefits
Competitive compensation and benefits, optional equity donation matching, generous vacation and parental leave, and flexible working hours.
Office space for collaborating with colleagues in San Francisco.