The whole posting in a few lines. Sign up to read it here and on every role you open.
Sign Up to ReadDescription
The company is a small AI infrastructure team that works on making inference for LLMs faster and more energy-efficient.
You will build and run systems that serve open-source models on many kinds of hardware, from low-level optimization through to production.
Responsibilities
- Bring new open-source models into service and tune each one for response time and how much it can process at once.
- Improve the serving stack through batching, KV cache handling, speculative decoding, and quantization.
- Write and tune GPU kernels in CUDA, HIP, and Triton for mixed hardware.
- Set up, deploy, and maintain clusters that combine hardware from several vendors.
- Keep production inference running reliably, with better monitoring and a lower cost per token.
- Talk with customers to learn what they need and fix their infrastructure problems.
Requirements
- You have 1 to 6 years in backend, infrastructure, or systems work, including low-level engineering beyond application code.
- You understand computer architecture, the memory hierarchy, and operating system internals well.
- You have worked with CUDA, Triton, or GPU programming. Experience with HIP and heterogeneous accelerators is useful.
- You know how to optimize ML inference, for example through quantization, batching, KV caching, or speculative decoding.
- You have deployed or run infrastructure at scale, such as clusters or large GPU fleets.
- You have strong computer science fundamentals and have shown ownership, fast learning, or achievement on hard technical problems.
- You are comfortable using AI coding tools to improve how engineering gets done and talking with customers.
- Early-career candidates and new graduates with relevant internships are welcome to apply.
Conditions
Base salary is $200,000 to $300,000 per year.
Visa sponsorship is available.
The role is on-site in San Francisco, California, 5 days a week.