You and the process

With a person
Anna, in-house recruiter
9 years hiring engineers
30 minutes with a real recruiter
They read your CV with you, on a call, and say where the offers are being lost.
Didn’t find what you were looking for? Tell us what to build
Early Window: Closes in
Company hidden

Staff Backend Engineer

  • Office
  • 6+ years

Salary

$250,000 - 350,000/ year

AI summary

For members

The whole posting in a few lines. Sign up to read it here and on every role you open.

Sign Up to Read

Description

Every LLM call those customers make runs through its code, so a failure there reaches them directly.

You will own key parts of the proxy: routing, auth, caching, spend tracking, guardrails and observability. The aim is correct, fast and dependable behavior at 1B+ requests a month.

Much of the work is tracking down silent failures and fixing root causes in code instead of adding another dashboard. You will also refactor components that keep failing while keeping their behavior unchanged.

The team is mostly early-career, and you will set the standard for how they write, test and ship backend code.

Responsibilities

  • Take end-to-end ownership of core proxy components, from design through production, including incidents.
  • Find and eliminate entire bug classes, such as race conditions, stale cache state across replicas, and errors swallowed before users see them.
  • Make sure every guardrail and policy check, such as applyguardrail, is logged, traceable and fails loudly when it should.
  • Build guardrails that run inside the gateway and block common attacks on LLM APIs, MCP servers and agents.
  • Keep Datadog, Splunk, Prometheus and OpenTelemetry integrations accurate at 1B+ requests a month, with little latency overhead and no memory leaks, including from Prometheus metric cardinality.
  • Help move performance-critical paths from Python to Rust without changing behavior.
  • Raise the team's standards for code review, testing and debugging.

Requirements

  • 5+ years building production backend systems, with real ownership of mission-critical work such as payments, gateways, core APIs, data stores or anything where downtime hurts customers.
  • Strong Python skills and experience with async or high-throughput services, such as asyncio or FastAPI.
  • A record of making production systems more reliable through code, via refactors, re-architectures, migrations or permanent root-cause fixes.
  • Excellent debugging across services, databases (Postgres, Redis) and distributed systems.
  • High ownership: you answer for the customer impact of what you ship.
  • BS or MS in Computer Science or a related field, or equivalent proof of exceptional ability.
  • Work in production with Rust, Go or C++.
  • Time at a fast-growing startup with a large, real codebase.
  • Experience with LLM APIs, proxies or API gateways.
  • Open-source contributions.

Conditions

Based onsite in San Francisco.

Base salary of $250-$350k.

Benefits

  • Equity.
  • Health, dental and vision coverage.

Where you’d work

From the office

The office

About the company

Company hidden

Office in San Francisco, United States

Your chances

Still hiring, not crowded yet, and you'd be among the first.

  • 17 checks run
  • 8 good signs
  • 1 red flag

Still hiring?

11 checks

Actively hiring

In its favour4

  • Still on the company's own careers site, checked 1 h agoModerate evidence
  • Found in the last 48 hours, before the big job boardsModerate evidence
  • Specific about the basics: pay, place, level, stack and contract all statedSlight evidence
1 moreFewer
  • States its salarySlight evidence

How crowded?

6 checks

Low

In its favour4

  • In its Early Window: not on the big job boards yetStrong evidence
  • In the office in San Francisco: only people nearby can take itSlight evidence
  • Senior level: far fewer people qualifySlight evidence
1 moreFewer
  • Asks for Redis, which only 1% of open roles doSlight evidence

Against it1

  • Pays more than 91% of similar roles: that draws applicantsSlight evidence

Fits Me

How well does this role fit you?

Answer a few questions or drop your CV, and every role gets a fit score with the reasons, this one first.

  • Your field
  • Level
  • Stack
  • Work model
  • Salary floor
  • Must-haves
Details12 facts · Role, Location, Compensation, Employment
Tech stack
  • Python
  • Go
  • PostgreSQL
  • C++
  • Rust
  • Redis
  • LLMs
Seniority
Staff
Type
Full-time
Equity
Equity offered
Specialty
Backend
Region
United States
Show 6 more factsShow less

Role

Category
Development
Specialty
Backend
Seniority
Staff
Experience
5+ years
Tech stack
  • Python
  • Go
  • PostgreSQL
  • C++
  • Rust
  • Redis
  • LLMs

Location

Work model
Office
Region
United States
Office
  • San Francisco, United States

Compensation

Salary
$250,000 - 350,000 / year
Pay period
Annual
Equity
Equity offered

Employment

Type
Full-time

Something wrong with this vacancy?

Similar vacancies

Share this vacancy

What's wrong with it?

The employer never sees who reported.

Reason