You and the process

With a person
Anna, in-house recruiter
9 years hiring engineers
30 minutes with a real recruiter
They read your CV with you, on a call, and say where the offers are being lost.
Didn’t find what you were looking for? Tell us what to build
Together AI

Engineering Manager, Kubernetes Bare Metal Infrastructure

  • Office
  • 6+ years

Salary

$300,000 - 350,000/ year

AI summary

For members

The whole posting in a few lines. Sign up to read it here and on every role you open.

Sign Up to Read

Description

You will lead the team that runs Kubernetes on bare-metal GPU infrastructure. The role is for an engineering manager or tech lead manager.

You will be accountable for how thousands of GPUs across our data centers are provisioned, orchestrated and kept healthy. Customers rely on these GPUs to train, fine-tune and serve models on the platform.

The company is an AI native cloud, a full-stack platform for running open and custom generative AI models, from GPU clusters through to serverless inference. Its research and engineering teams publish widely used open-source work, including FlashAttention.

The team sits at the base of that stack. Training runs, inference endpoints and customer clusters all depend on the reliability, utilization and speed of the bare-metal Kubernetes layer.

Responsibilities

  • Set the technical direction for running bare-metal Kubernetes across several data centers and regions.
  • Build zero-touch provisioning for GPU nodes, covering firmware, OS imaging, cluster join and validation.
  • Manage the full cluster lifecycle at scale, including upgrades, capacity growth, node health checks, automated remediation and decommissioning.
  • Make GPU scheduling, topology-aware placement and high-performance networking (InfiniBand, RoCE, RDMA) core parts of the platform.
  • Define and meet SLOs for cluster availability, node bring-up time and GPU utilization.
  • Extend observability across hardware, networking, Kubernetes and workloads so faults are caught before customers notice them.
  • Shorten the time it takes to go from racked hardware to customer-ready capacity.
  • Hire, manage and develop infrastructure and platform engineers, holding them to high standards of technical depth and operational ownership.
  • Plan priorities and delivery, balancing roadmap work against fleet growth and production support.
  • Foster a sustainable on-call culture with blameless postmortems and follow-through on remediation.
  • Coordinate with data center operations, networking, hardware vendors, and inference, training and product teams on capacity planning and new hardware generations.
  • Explain the platform's needs and trade-offs to leadership, and turn customer requirements into a clear roadmap.

Requirements

  • 8+ years in infrastructure, SRE or platform engineering, including 2+ years managing engineers.
  • Deep hands-on experience running Kubernetes in production at scale, ideally on bare metal and not only through managed services.
  • Solid understanding of Linux, networking and hardware, including kernels, drivers, BIOS/BMC, PXE boot and out-of-band management.
  • Experience building fleet provisioning and lifecycle automation with tools such as Cluster API, Metal3, Ironic, MAAS, Ansible or Terraform.
  • A record of owning production systems with on-call duties, SLOs and incident response.
  • Sound technical judgment: you can review designs, debug alongside your team and explain trade-offs clearly.
  • Clear communication with engineers, leadership and external partners.
  • Experience operating large NVIDIA GPU clusters (H100, H200, B200, GB200) along with the NVIDIA GPU and Network Operators, DCGM and NCCL.
  • Familiarity with InfiniBand, RoCE, SR-IOV and high-performance storage for AI workloads.
  • Experience with Slurm, Kueue, Volcano or other batch schedulers for training workloads.
  • Contributions to Kubernetes or related CNCF projects.
  • Experience at an AI lab, GPU cloud or hyperscaler.

Conditions

This is a full-time position with a US base salary range of $300,000 to $350,000, plus equity and benefits.

Salary ranges depend on location, level and role. Individual pay is set by experience, skills and job-related knowledge.

The company serves over 400+ trillion tokens a month.

Benefits

  • Competitive compensation, startup equity, health insurance and other competitive benefits.

Where you’d work

From the office

The office

About the company

Together AI

  • Industry: AI

Office in San Francisco, United States

Also hiring in Amsterdam, Netherlands

1 of their 14 open roles is remote

Your chances

Worth a look before you spend an evening tailoring a CV for it.

  • 21 checks run
  • 1 red flag

Still hiring?

14 checks

No red flags

How crowded?

7 checks

1 red flag

Fits Me

How well does this role fit you?

Answer a few questions or drop your CV, and every role gets a fit score with the reasons, this one first.

  • Your field
  • Level
  • Stack
  • Work model
  • Salary floor
  • Must-haves
Details13 facts · Role, Location, Compensation, Employment, Company
Tech stack
  • Kubernetes
  • Terraform
  • Linux
  • Ansible
Seniority
Manager
Type
Full-time
Equity
Equity offered
Industry
AI
Specialty
DevOps
Show 7 more factsShow less

Role

Category
DevOps & Infrastructure
Specialty
DevOps
Seniority
Manager
Experience
8+ years
Tech stack
  • Kubernetes
  • Terraform
  • Linux
  • Ansible

Location

Work model
Office
Region
United States
Office
  • San Francisco, United States

Compensation

Salary
$300,000 - 350,000 / year
Pay period
Annual
Equity
Equity offered

Employment

Type
Full-time

Company

Industry
AI

Something wrong with this vacancy?

Similar vacancies

Share this vacancy

What's wrong with it?

The employer never sees who reported.

Reason