You and the process

With a person
Anna, in-house recruiter
9 years hiring engineers
30 minutes with a real recruiter
They read your CV with you, on a call, and say where the offers are being lost.
Didn’t find what you were looking for? Tell us what to build
LiteLLM

ML Engineer

  • Office
  • 0-3 years

Salary

$200,000 - 250,000/ year

AI summary

For members

The whole posting in a few lines. Sign up to read it here and on every role you open.

Sign Up to Read

Description

The company makes the company, an AI gateway used by large companies. It provides a unified API for more than 100 LLM providers, with routing, authentication, budgets, observability and governance for production AI systems. The team is now building a second product.

This role focuses on the work below.

Responsibilities

  • Make sure the company presents one calling format for LLM APIs that follows the OpenAI and Anthropic specs.
  • Write transformations that convert OpenAI and Anthropic API requests into the formats used by different LLM providers.
  • Build provider-agnostic features, such as session management for non-OpenAI models behind the /v1/responses API.
  • Design how provider-level features such as skills, compaction and batches are supported at the gateway level.
  • Improve the developer experience for the millions of users who reach this work through the company Python SDK.
  • Maintain a strong LLM interoperability layer across LLMs, MCP and agents, including a range of MCP authentication flows and agent use cases.
  • Work directly with the CEO and CTO on critical projects.

Requirements

  • Have 2 years of backend or full-stack experience with production systems.
  • Know Python, FastAPI, Redis and Postgres, which make up the tech stack.
  • Have worked with the OpenAI API and understand the difference between /chat/completions and /responses, including API-specific details.
  • Care about open source and about how users engage with the product.
  • Be willing to talk with users and help them solve real problems.
  • Work hard and do well in a small team.

Conditions

The role is based on the team's current setup, with no further details given in the posting.

Benefits

  • Work directly with the founders every day.
  • Help build products used by thousands of engineering teams worldwide.
  • Make real product decisions, not only implementation work.
  • See your work reach production quickly.
  • Help shape the future of AI infrastructure and developer tooling.
  • Competitive salary, equity, and health, dental and vision benefits.

Where you’d work

From the office

The office

About the company

LiteLLM

Office in San Francisco, United States

Your chances

Worth a look before you spend an evening tailoring a CV for it.

  • 21 checks run
  • 1 red flag

Still hiring?

14 checks

No red flags

How crowded?

7 checks

1 red flag

Fits Me

How well does this role fit you?

Answer a few questions or drop your CV, and every role gets a fit score with the reasons, this one first.

  • Your field
  • Level
  • Stack
  • Work model
  • Salary floor
  • Must-haves
Details11 facts · Role, Location, Compensation, Employment
Tech stack
  • Python
  • PostgreSQL
  • LLMs
Type
Full-time
Equity
Equity offered
Specialty
ML
Region
United States
Pay period
Annual
Show 5 more factsShow less

Role

Category
Data & Analytics
Specialty
ML
Experience
2+ years
Tech stack
  • Python
  • PostgreSQL
  • LLMs

Location

Work model
Office
Region
United States
Office
  • San Francisco, United States

Compensation

Salary
$200,000 - 250,000 / year
Pay period
Annual
Equity
Equity offered

Employment

Type
Full-time

Something wrong with this vacancy?

Similar vacancies

Share this vacancy

What's wrong with it?

The employer never sees who reported.

Reason