You and the process

With a person
Anna, in-house recruiter
9 years hiring engineers
30 minutes with a real recruiter
They read your CV with you, on a call, and say where the offers are being lost.
Didn’t find what you were looking for? Tell us what to build
LiteLLM

Staff Engineer, Systems and Reliability

  • Office
  • 6+ years

Salary

$250,000 - 350,000/ year

AI summary

For members

The whole posting in a few lines. Sign up to read it here and on every role you open.

Sign Up to Read

Description

The company runs the most widely used AI gateway, trusted by companies such as Adobe, Netflix, and NASA. It offers a unified API for over 100+ LLM providers with routing, authentication, budgets, observability, and governance, and thousands of companies run production AI on it.

ARR grew 6x in the last year to nearly $30M, and the team has 20 people.

This is a hands-on Staff role focused on the stability and reliability of the gateway. You will find the components behind regressions and security issues and refactor or rewrite them so the same problem stops recurring across the codebase. Quality rises through your own code and reviews, not process documents.

The gateway is critical infrastructure for customers. When it fails, their AI products fail too, so your job is to reduce those failures and catch problems before customers do.

The tech stack is Python, FastAPI, Redis, and Postgres, and the gateway is moving to Rust.

You will work with the security and infrastructure team and report to the CEO, later to the Head of Engineering. The role is in person in San Francisco.

Responsibilities

  • Trace regressions and security issues to their root causes and fix the patterns that produce them.
  • Lead refactors, rewrites, and migrations of core product code that customers cannot afford to have fail.
  • Define what makes the test suite trustworthy and build it.
  • Keep CI and releases fast and reliable with a large open source contributor base.
  • Review the most important code paths and raise the bar through your own code and review.
  • Lift quality across the team without becoming a bottleneck.

Requirements

  • 3+ years of experience, with scope growing quickly at a strong engineering company. The scope you have owned matters more than the Staff title.
  • You have owned stability or reliability at your current company.
  • You have owned mission-critical production systems and led rewrites or code migrations.
  • You can enter an unfamiliar codebase and find the real root cause.
  • You reason through hard bugs and large refactors yourself, and you review AI-written code as carefully as human-written code.
  • You consider how a change affects users before shipping it.
  • You have strong software engineering fundamentals, preferably in Python.
  • No security background is needed. The security team finds issues, and you fix the patterns behind them.
  • Bonus points: Rust experience.
  • Bonus points: you have kept CI healthy on an open source project with heavy outside contribution.
  • Bonus points: experience at an early-stage startup.

Conditions

In person in San Francisco.

Relocation help is available, and H-1B transfers are supported.

Reports to the CEO, later to the Head of Engineering.

Benefits

  • Work directly with the founders every day.
  • Own reliability for infrastructure that thousands of companies run in production.
  • Own a whole area of the codebase. There are no PMs, so engineers talk to customers and decide what to build.
  • Your work ships in open source and is used by engineering teams worldwide.
  • Move quickly and see your changes in production right away.
  • Competitive salary, equity, and health, dental, and vision benefits.

Where you’d work

From the office

The office

Relocation offered

Visa sponsored

About the company

LiteLLM

Office in San Francisco, United States

Your chances

Worth a look before you spend an evening tailoring a CV for it.

  • 23 checks run
  • 3 red flags

Still hiring?

14 checks

No red flags

How crowded?

9 checks

3 red flags

Fits Me

How well does this role fit you?

Answer a few questions or drop your CV, and every role gets a fit score with the reasons, this one first.

  • Your field
  • Level
  • Stack
  • Work model
  • Salary floor
  • Must-haves
Details14 facts · Role, Location, Compensation, Employment
Tech stack
  • Python
  • PostgreSQL
  • Rust
  • Redis
  • LLMs
Seniority
Staff
Type
Full-time
Equity
Equity offered
Specialty
Backend
Region
United States
Show 8 more factsShow less

Role

Category
Development
Specialty
Backend
Seniority
Staff
Experience
3+ years
Tech stack
  • Python
  • PostgreSQL
  • Rust
  • Redis
  • LLMs

Location

Work model
Office
Region
United States
Office
  • San Francisco, United States
Relocation
Offered
Visa sponsorship
Sponsored

Compensation

Salary
$250,000 - 350,000 / year
Pay period
Annual
Equity
Equity offered

Employment

Type
Full-time

Something wrong with this vacancy?

Similar vacancies

Share this vacancy

What's wrong with it?

The employer never sees who reported.

Reason