You and the process

With a person
Anna, in-house recruiter
9 years hiring engineers
30 minutes with a real recruiter
They read your CV with you, on a call, and say where the offers are being lost.
Didn’t find what you were looking for? Tell us what to build
LiteLLM

Staff Engineer, AI

  • Office
  • 6+ years

Salary

$250,000 - 350,000/ year

AI summary

For members

The whole posting in a few lines. Sign up to read it here and on every role you open.

Sign Up to Read

Description

The company runs the most widely used AI gateway, trusted by Adobe, Netflix and NASA.

ARR grew 6x over the past year to nearly $30M, and the team has 20 people.

Developers reach 100+ model providers with a single interface.

The team is hiring a Staff Engineer to head the unified API group and make sure every new model and provider is supported from launch day.

This is a hands-on technical lead position. You will shape how the gateway unifies LLM APIs across the OpenAI and Anthropic specifications, mentor the engineers building it, and define quality standards.

Providers release new APIs, modalities and features every week. The goal is to support each one the day it appears, shrink the backlog of open source translation issues, and keep developers confident the gateway keeps pace.

You will lead the open source pod, where most issues concern LLM translation. Tech stack: Python, FastAPI, Redis and Postgres, with the gateway being moved to Rust.

Responsibilities

  • Lead the unified API team and carry each new model, endpoint and provider launch from start to finish.
  • Design gateway-level features that work across providers, for example session handling for non-OpenAI models.
  • Sort open source issues and external pull requests, including new providers and translation fixes, then merge, review or replace them with an internal fix.
  • Review the AI team's pull requests and enforce high code quality, including code produced with AI assistance.
  • Identify heavy users in your area and use their needs to choose what to build next.
  • Start new unification projects with a customer who will use them and report back.
  • Own the developer experience for the Python SDK and proxy.

Requirements

  • You have worked as a machine learning engineer and now build applied AI products.
  • You have 3+ years of experience, and the scope you have owned counts for more than a Staff title.
  • You know LLM APIs deeply, including how /chat/completions differs from /responses and the quirks of each provider.
  • You have built or maintained a unified API placed in front of many providers.
  • You have good developer experience judgment and can explain why an API or playground falls short.
  • You care about customers: you ask why someone wants a feature and how large the use case is before building it.
  • You can lead a team from day one and set its technical standard.
  • You have solid Python software engineering fundamentals.
  • Bonus points: experience at a company built on a unified API, such as OpenRouter, Vercel AI SDK, Merge, Segment or Fivetran.
  • Bonus points: you have built your own unified API or written about what a good one should do.
  • Bonus points: you have built evals for LLMs.
  • Bonus points: you have run the gateway in production.
  • Bonus points: open source contributions, especially to LLM tooling.

Conditions

This is an in-person role in San Francisco.

Relocation help is available, and the company supports H-1B transfers.

You report to the CEO at first, then to the Head of Engineering later.

Benefits

  • Work directly with the founders every day.
  • Lead the team behind the core product, the unified API that thousands of engineering teams depend on.
  • Own a full product area. There are no product managers, so engineers talk to customers and decide what gets built.
  • Your work ships as open source and is used by engineering teams around the world.
  • Changes reach production quickly, so you see the results right away.
  • Competitive salary, equity, and health, dental and vision benefits.

Where you’d work

From the office

The office

Relocation offered

Visa sponsored

About the company

LiteLLM

Office in San Francisco, United States

Your chances

Worth a look before you spend an evening tailoring a CV for it.

  • 23 checks run
  • 3 red flags

Still hiring?

14 checks

No red flags

How crowded?

9 checks

3 red flags

Fits Me

How well does this role fit you?

Answer a few questions or drop your CV, and every role gets a fit score with the reasons, this one first.

  • Your field
  • Level
  • Stack
  • Work model
  • Salary floor
  • Must-haves
Details14 facts · Role, Location, Compensation, Employment
Tech stack
  • Python
  • PostgreSQL
  • Rust
  • Redis
  • LLMs
Seniority
Staff
Type
Full-time
Equity
Equity offered
Specialty
Backend
Region
United States
Show 8 more factsShow less

Role

Category
Development
Specialty
Backend
Seniority
Staff
Experience
3+ years
Tech stack
  • Python
  • PostgreSQL
  • Rust
  • Redis
  • LLMs

Location

Work model
Office
Region
United States
Office
  • San Francisco, United States
Relocation
Offered
Visa sponsorship
Sponsored

Compensation

Salary
$250,000 - 350,000 / year
Pay period
Annual
Equity
Equity offered

Employment

Type
Full-time

Something wrong with this vacancy?

Similar vacancies

Share this vacancy

What's wrong with it?

The employer never sees who reported.

Reason