The company manages one of the largest and fastest-growing infrastructure fleets in the industry, spanning multiple accelerator families, CPU families and cloud providers.
The Capacity Engineering team makes sure every infrastructure resource is accounted for, well used and efficiently allocated. The team owns the data, tooling and operations that let the company plan, measure and maximize utilization across first-party and third-party compute.
This is a pipeline role feeding four areas: data platform, planning, efficiency, and attribution and forecasting. You will focus mainly on one, depending on your background and business priority, but the boundaries overlap.
Data platform covers pipelines that ingest occupancy and utilization telemetry from Kubernetes clusters, normalize billing and usage across cloud providers, and serve the BigQuery tables used across the org. Correctness, completeness and latency matter most. Consumers range from research engineers to finance and leadership, so you will define schema contracts, make data discoverable and work out what people need.
Planning covers knowing what the fleet has, where it is going and what blocks it. It includes real-time cluster health tooling, capacity planning platforms, alerts on occupancy drops and allocation problems, and fixes to scheduling and fragmentation. You will work on Kubernetes operations and cross-team coordination.
Efficiency covers measuring how well each major workload uses its hardware across training, inference and eval. It includes benchmarking infrastructure, per-config baselines, and working with system-owning teams to close gaps. The metric must be accurate enough that the team it measures agrees with it.
Attribution and forecasting connects fleet cost to business activity. It involves reconciling CSP billing exports against vendor telemetry and internal systems with mismatched schemas, attributing spend to the workloads and teams behind it, and turning inference demand and research roadmaps into a defensible compute plan. Efficiency metrics must hold up with finance: free of pure demand and unit-price effects, reproducible month to month, and clear to a CFO.
Responsibilities
Build the planning and allocation stack: tools leadership uses to allocate capacity, teams use to plan against their allocations, and the scheduler enforces. This includes cross-region and cross-provider placement, guardrails, queueing and occupancy KPIs.
Lead efficiency programs covering stranding, rightsizing, recovery of unused capacity, and job-level utilization across training, inference and eval. Set per-config baselines and work with system-owning teams to close gaps. Utilization gains are worth very large sums at this scale.
Own attribution and forecasting. Reconcile billing from more than ten providers against telemetry and internal systems, attribute spend to the workloads that create it, and turn demand signals and research roadmaps into a defensible compute plan and supply pipeline.
Build the data platform underneath this work: pipelines that bring occupancy, utilization and cost from a fast-changing fleet into BigQuery. You own completeness, latency SLOs and gap detection. Each new provider is a new integration.
Run Kubernetes-native systems at scale, including collection agents, workload labeling, and the taint, reservation and scheduling behavior that decides which capacity is usable.
Treat the output as a product. Gather your own requirements, define schema contracts, and design for consumers from research engineers to a CFO. You own on-call and SLOs, because these surfaces are critical to the company.
Requirements
A strong record of building and operating production systems. This is a hands-on engineering role with a devops focus.
Production-quality Python and SQL. Most pipeline code is Python, and the presentation layer is BigQuery SQL, including table-valued functions and views. Both must be idiomatic, well tested and maintainable.
Deep experience with at least one major cloud provider (Amazon Web Services, Google Cloud or Microsoft Azure) and its operations.
Experience with Prometheus, PromQL and Grafana, including writing recording rules and building monitoring that engineering teams rely on.
Ability to gather your own requirements, work across organizational boundaries, and operate in an ambiguous environment with limited direction.
Preferred: experience with capacity planning, resource management or cost attribution at a hyperscaler or in a large-scale machine learning environment. Product engineering and developer experience work counts.
Preferred: experience with scheduling and packing efficiency, or profiling-driven optimization of large distributed workloads.
Preferred: multi-cloud data ingestion, especially normalizing billing exports, reservation APIs, on-demand capacity reservations, commitments and vendor telemetry from providers with different billing arrangements.
Preferred: total cost of ownership and forecasting, including determining whether infrastructure growth is caused by or merely correlated with business drivers.
Preferred: accelerator infrastructure knowledge, such as GPU metrics (DCGM), TPU utilization, Trainium power and utilization metrics, or machine learning training and inference systems at the hardware level.
Preferred: building internal data products with self-service access, schema contracts, API serving, documentation and discoverability, with attention to how the data is consumed.
Preferred: storage efficiency, retention and lifecycle programs at exabyte scale.
Conditions
Annual salary range: $320,000 to $485,000 USD.
Minimum education: a Bachelor's degree or equivalent combination of education, training and experience. Required field of study: a field relevant to the role, shown through coursework, training or professional experience. Required years of experience match the internal job level for the position.
Hybrid policy: staff are expected to be in an office at least 25% of the time. Some roles may require more office time.
Visa sponsorship is available, though not for every role or candidate. If an offer is made, the company will make every reasonable effort to obtain a visa and retains an immigration lawyer to help.
Candidates are encouraged to apply even if they do not meet every qualification. The company aims for a diverse team and welcomes applicants from underrepresented groups.