You and the process

With a person
Anna, in-house recruiter
9 years hiring engineers
30 minutes with a real recruiter
They read your CV with you, on a call, and say where the offers are being lost.
Didn’t find what you were looking for? Tell us what to build
Early Window: Closes in
Company hidden

Staff+ Software Engineer

  • Hybrid
  • 6+ years

Salary

$320,000 - 485,000/ year

AI summary

For members

The whole posting in a few lines. Sign up to read it here and on every role you open.

Sign Up to Read

Description

The company runs one of the largest AI fleets in the industry, spanning several accelerator families, CPU families, and cloud providers. It is a fast-growing group of researchers, engineers, policy experts, and business leaders.

The Capacity Engineering team makes sure every infrastructure resource is accounted for, well used, and allocated efficiently. It owns the data, tooling, and operations used to plan, measure, and raise utilization across first-party and third-party compute.

You will build the production systems behind this work: pipelines that ingest and normalize telemetry from varied cloud environments, observability tools that show real-time fleet health, and instrumentation that measures how efficiently each major workload uses its hardware. You will write production-quality code every day, run Kubernetes-native infrastructure at real scale, and shape capacity decisions.

You will work with research engineering, infrastructure, inference, and finance teams. The role moves between data engineering, systems engineering, and observability, in a setting with high autonomy and high ambiguity.

This is a pipeline role with four areas. Your background and the business priorities decide where you focus, but the areas overlap:

Data platform: pipelines that pull occupancy and utilization telemetry from Kubernetes clusters, normalize billing and usage across cloud providers, and serve BigQuery tables to the whole org. Correctness, completeness, and latency matter most. Consumers range from research engineers to finance and leadership, so you will define schema contracts, make data easy to find, and work out what people need.

Planning: knowing what the fleet has, where it is heading, and what blocks it. Making fleet state clear and actionable in real time through cluster health tools, capacity planning platforms, alerts on occupancy drops and allocation problems, and fixes to scheduling and fragmentation. This covers Kubernetes operations on one side and cross-team coordination on the other.

Efficiency: measuring and improving how well each major workload uses its hardware. You will instrument utilization across training, inference, and eval systems, build benchmarking infrastructure, set baselines per config, and work with system-owning teams to close gaps.

Attribution and forecasting: linking what the fleet costs to what the business does with it. You will reconcile CSP billing exports against vendor telemetry and internal systems with mismatched schemas, assign spend to the workloads and teams behind it, and turn inference demand and research roadmaps into a defensible compute plan. Efficiency metrics must hold up with finance: free of pure demand and unit-price effects, reproducible month to month, and clear to a CFO.

Responsibilities

  • Build the planning and allocation stack that leadership uses to assign capacity, teams use to plan against their allocations, and the scheduler enforces. This includes cross-region and cross-provider placement, guardrails, queueing, and occupancy KPIs.
  • Lead efficiency programs covering stranded capacity, rightsizing, recovery of unused capacity, and job-level utilization across training, inference, and eval. Set per-config baselines and work with system-owning teams to close gaps. You will influence decisions about a major area of company investment.
  • Own attribution and forecasting. Reconcile billing from 10+ providers against telemetry and internal systems, attribute spend to the workloads that generate it, and turn demand signals and research roadmaps into a defensible compute plan and supply pipeline.
  • Build the data platform under all of this: pipelines that load occupancy, utilization, and cost from a fast-diversifying fleet into BigQuery. You own completeness, latency SLOs, and gap detection. Every new provider is a new integration.
  • Operate Kubernetes-native systems at scale, including collection agents, workload labeling, and the taint, reservation, and scheduling behavior that determines which capacity is actually usable.
  • Treat the output as a product. Gather your own requirements, define schema contracts, and design for consumers from research engineers to a CFO. You own on-call duties and SLOs, since these surfaces are critical to the company.

Requirements

  • A strong record of building and operating production systems. This is a hands-on engineering role with a devops focus.
  • Python and SQL at production quality. Most pipeline code is Python, and the presentation layer is BigQuery SQL, including table-valued functions and views. Code must be idiomatic, well tested, and maintainable.
  • Deep experience with one major cloud provider (Amazon Web Services, Google Cloud, or Microsoft Azure) and its operations.
  • Experience with the observability stack, including Prometheus, PromQL, and Grafana, with recording rules and monitoring that engineering teams rely on.
  • Ability to set your own requirements and work across organizational boundaries in an ambiguous setting with limited direction.
  • Preferred: experience with capacity planning, resource management, or cost attribution at a hyperscaler or in a large-scale machine learning environment. Product engineering and developer experience work counts.
  • Preferred: scheduling and packing efficiency, or profiling-driven optimization of large distributed workloads.
  • Preferred: multi-cloud data ingestion, especially normalizing billing exports, reservation APIs, on-demand capacity reservations, commitments, and vendor telemetry across providers with different billing models.
  • Preferred: total cost of ownership and forecasting, including determining whether infrastructure growth is caused by or merely correlated with business drivers.
  • Preferred: accelerator familiarity, such as DCGM GPU metrics, TPU utilization, Trainium power and utilization metrics, or experience with machine learning training and inference at the hardware level.
  • Preferred: experience building internal data products with self-service access, schema contracts, API serving, documentation, and discoverability. Thinking about how data gets consumed, not only how it is pipelined.
  • Preferred: storage efficiency, retention, and lifecycle program experience at very large scale.

Conditions

Annual salary range: $320,000 - $485,000 USD.

Minimum education: a bachelor's degree or an equivalent mix of education, training, and experience. The field of study should be relevant to the role, shown through coursework, training, or professional work. Required years of experience follow the internal job level.

Hybrid policy: staff are currently expected in an office at least 25% of the time. Some roles may require more.

Visa sponsorship is available, but not for every role and every candidate. If an offer is made, the company will make every reasonable effort to secure a visa and retains an immigration lawyer to help.

Apply even if you do not meet every qualification. The team values a range of perspectives.

Benefits

  • Competitive compensation and benefits.
  • Optional matching of equity donations.
  • Generous vacation and parental leave.
  • Flexible working hours.
  • An office space for collaborating with colleagues.

Where you’d work

Part of the week in the office

About the company

Company hidden

  • Industry: AI

Offices in San Francisco, United States, New York City, United States, Seattle, United States

Your chances

Still hiring, not crowded yet, and you'd be among the first.

  • 18 checks run
  • 6 good signs
  • 3 red flags

Still hiring?

11 checks

Actively hiring

In its favour4

  • Still on the company's own careers site, checked 1 h agoModerate evidence
  • Found in the last 48 hours, before the big job boardsModerate evidence
  • Specific about the basics: pay, place, level, stack and contract all statedSlight evidence
1 moreFewer
  • States its salarySlight evidence

Against it1

  • Listed again after an earlier listing of the same role closedModerate evidence

How crowded?

7 checks

Low

In its favour2

  • In its Early Window: not on the big job boards yetStrong evidence
  • Senior level: far fewer people qualifySlight evidence

Against it2

  • Sponsors visas: applicants from abroad compete for it tooSlight evidence
  • Pays more than 98% of similar roles: that draws applicantsSlight evidence

Fits Me

How well does this role fit you?

Answer a few questions or drop your CV, and every role gets a fit score with the reasons, this one first.

  • Your field
  • Level
  • Stack
  • Work model
  • Salary floor
  • Must-haves
Details14 facts · Role, Location, Compensation, Employment, Company
Tech stack
  • Python
  • SQL
  • AWS
  • Google Cloud
  • Kubernetes
  • Azure
Seniority
Staff
Type
Full-time
Equity
Equity offered
Industry
AI
Specialty
Backend
Show 8 more factsShow less

Role

Category
Development
Specialty
Backend
Seniority
Staff
Experience
6+ years
Tech stack
  • Python
  • SQL
  • AWS
  • Google Cloud
  • Kubernetes
  • Azure

Location

Work model
Hybrid
Region
United States
Offices
  • San Francisco, United States
  • New York City, United States
  • Seattle, United States
Visa sponsorship
Sponsored

Compensation

Salary
$320,000 - 485,000 / year
Pay period
Annual
Equity
Equity offered

Employment

Type
Full-time

Company

Industry
AI

Something wrong with this vacancy?

Similar vacancies

Share this vacancy

What's wrong with it?

The employer never sees who reported.

Reason