You and the process

With a person
Anna, in-house recruiter
9 years hiring engineers
30 minutes with a real recruiter
They read your CV with you, on a call, and say where the offers are being lost.
Didn’t find what you were looking for? Tell us what to build
Early Window: Closes in
Company hidden

Machine Learning Infrastructure Engineer

  • Office

Salary

$200,000 - 400,000/ year

AI summary

For members

The whole posting in a few lines. Sign up to read it here and on every role you open.

Sign Up to Read

Description

The team builds large physics foundation models that predict how real-world systems behave and help identify effective actions.

You will build the distributed infrastructure that supports model training and inference, with weather data as a major application.

Responsibilities

  • Set up, ship, and keep running distributed clusters for ML training and inference.
  • Build scalable end-to-end pipelines for datasets at petabyte scale and for model training across the full ML lifecycle.
  • Assess training methods, including parallelization approaches and the trade-offs in numerical precision across different model sizes.
  • Profile and debug low-level GPU operations to make them faster.
  • Follow current research and bring promising ideas into infrastructure and workflows.

Requirements

  • Between 2 and 10 years building large-scale infrastructure for pretrained or foundation models, covering compute clusters, data systems, inference, and distributed training.
  • Hands-on work speeding up large training, data, or inference workloads, with experience in distributed training frameworks such as FSDP or DeepSpeed.
  • Experience analyzing and debugging GPU performance at a low level, including CUDA and JAX.
  • Working knowledge of Python, C++, Linux, NVIDIA GPUs, cloud platforms such as GCP, AWS, or Azure, and containers and orchestration with Docker and Kubernetes.
  • Background in physical AI or AI for science, such as robotics, autonomous systems, or biology, and a drive to work on scientific or physical-world problems.
  • Able to work across the ML lifecycle at a fast pace, including analyzing experiments and ablation studies.

Conditions

Base salary ranges from $200,000 to $400,000 per year.

Based on-site in San Francisco, California, with relocation support provided.

Visa sponsorship is not offered.

Where you’d work

From the office

The office

Relocation offered

No visa sponsorship

About the company

Company hidden

Office in San Francisco, United States

Your chances

Still hiring, not crowded yet, and you'd be among the first.

  • 18 checks run
  • 6 good signs
  • 3 red flags

Still hiring?

11 checks

Actively hiring

In its favour3

  • Still on the company's own careers site, checked 1 h agoModerate evidence
  • Found in the last 48 hours, before the big job boardsModerate evidence
  • States its salarySlight evidence

Against it1

  • Listed for the 5th time after earlier listings of it closedModerate evidence

How crowded?

7 checks

Low

In its favour3

  • In its Early Window: not on the big job boards yetStrong evidence
  • In the office in San Francisco: only people nearby can take itSlight evidence
  • Asks for CUDA, which only 1% of open roles doSlight evidence

Against it2

  • Offers relocation: people from other countries apply tooSlight evidence
  • Pays more than 93% of similar roles: that draws applicantsSlight evidence

Fits Me

How well does this role fit you?

Answer a few questions or drop your CV, and every role gets a fit score with the reasons, this one first.

  • Your field
  • Level
  • Stack
  • Work model
  • Salary floor
  • Must-haves
Details11 facts · Role, Location, Compensation, Employment
Tech stack
  • Python
  • AWS
  • Kubernetes
  • Docker
  • Azure
  • CUDA
Type
Full-time
Specialty
ML
Region
United States
Pay period
Annual
Show 6 more factsShow less

Role

Category
Data & Analytics
Specialty
ML
Tech stack
  • Python
  • AWS
  • Kubernetes
  • Docker
  • Azure
  • CUDA

Location

Work model
Office
Region
United States
Office
  • San Francisco, United States
Relocation
Offered
Visa sponsorship
Not sponsored

Compensation

Salary
$200,000 - 400,000 / year
Pay period
Annual

Employment

Type
Full-time

Something wrong with this vacancy?

Similar vacancies

Share this vacancy

What's wrong with it?

The employer never sees who reported.

Reason