Working Student - LLM Inference Performance

Description

The company builds low-power, high-performance generative AI inference systems, combining custom silicon, hardware and software for hyperscaler and neocloud data center customers.

It is a well funded startup. The team works across Sunnyvale, CA, Munich, Germany, and remotely across North America and Europe.

The role centres on one question: how fast should an LLM building block run on the hardware, how fast does it run now, and what would speed it up?

You will work day to day with hardware and software engineers, using the Rust stack, the Python SDK and AI coding agents.

The team values putting people first, ethics and integrity, ambitious goals, excellence, ownership and results, and respecting differences between people.

Responsibilities

• Learn how the silicon handles data, memory and synchronization, and how attention, MLP and Mixture-of-Experts (MoE) routing map onto it.
• Write, test and tune low-level kernels that speed up LLM inference.
• Build analytical and roofline models, check them against simulator and real-silicon results, and find performance bottlenecks.
• Create benchmark harnesses, profiling scripts and trace visualizations that make performance easier to measure and improve.
• Explore how AI coding agents can help with kernel work, optimization and benchmarking, and which tools and context they need.
• Work with compiler, runtime and kernel engineers to turn performance findings into practical changes.
• Example project: build a benchmark and profiling harness for a building block such as decode attention at two abstraction levels, compare it with theoretical estimates, and make it usable by AI agents.

Requirements

• Currently enrolled in a Master's degree program in computer science, electrical engineering, mathematics or a related technical field.
• Strong Python skills and at least one of C, C++ or Rust.
• Understanding of computer architecture, memory systems, bandwidth and parallel computing, gained through coursework or personal projects.
• Interest in low-level programming, hardware performance and optimization.
• Practical experience using AI coding agents, with the ability to judge and improve their output.
• An experimental mindset: you form hypotheses, analyze measurements and communicate findings clearly.
• Curiosity, initiative and willingness to learn new technologies.
• Plus: GPU or AI accelerator programming.
• Plus: embedded systems or bare-metal programming.
• Plus: performance profiling, benchmarking or optimization.
• Plus: LLM inference fundamentals and architectures.

Conditions

Working student position.

Based in Sunnyvale, CA or Munich, Germany, or remote from across North America and Europe.

Benefits

• Comprehensive benefits, competitive compensation, flexible spending options and recognition programs.
• Work on next-generation AI inference hardware and software.
• Apply your university knowledge to real silicon and difficult engineering problems.
• Own meaningful projects that directly support engineering teams.
• Learn from specialists in hardware architecture, compilers, runtimes and performance optimization.
• Gain practical experience building with AI coding agents and shaping how they work with advanced hardware.
• Help build performance tools and workflows for current and future generations of the company's technology.