Member of Technical Staff

Description

About the Role

Join a small AI infrastructure team building systems that improve the performance and efficiency of large language model inference. You will work across software and hardware, with broad ownership from customer needs through production deployment.

Responsibilities

• Deliver support for new open-source models, tuning them for latency and throughput.
• Optimize inference serving, including batching, KV caching, speculative decoding, and quantization.
• Write and tune kernels using CUDA, HIP, and Triton across a range of accelerator platforms.
• Design, deploy, and operate clusters spanning multiple hardware vendors.
• Run production inference across heterogeneous fleets, improving reliability, observability, and cost efficiency.
• Work with customers to understand problems and develop practical systems solutions.

Requirements

• Typically 1 to 6 years of relevant experience, including strong early-career candidates with substantial internship or project experience.
• Experience in backend, infrastructure, or systems engineering, with demonstrated work beyond application-level development.
• Strong understanding of computer architecture, memory hierarchy, and operating system internals.
• Familiarity with GPU programming or inference optimization, such as CUDA, Triton, quantization, batching, KV caching, or speculative decoding.
• Experience deploying or operating infrastructure at scale, such as clusters or large GPU fleets, is valuable.
• Strong computer science fundamentals, ownership of challenging technical work, and effective use of AI coding tools.
• Comfort communicating with customers and working across a broad range of engineering problems.

Conditions

Base salary: $200,000 to $300,000 annually. Visa sponsorship is available.

On-site in San Francisco, California, five days per week.