Working Student - LLM Quantization & Inference Performance

Description

The company builds generative AI inference systems that pair custom silicon with hardware and software. The systems aim for high performance and low power and serve hyperscaler and neocloud data center customers.

The company is a well funded startup with offices in Sunnyvale, CA and Munich, Germany. Other team members work remotely across North America and Europe.

The role centers on two questions: how much numerical precision an LLM needs and where in the architecture precision matters most. It also asks when quantization speeds up runtime and what it costs in model quality.

You will turn these questions into experiments and turn experiments into maintainable code. Your findings will shape how models are optimized for the company's hardware, working alongside the quantization, compiler and runtime engineers.

The company values putting people first, ethics and integrity, thinking big, aiming for excellence, owning results, making each person stronger through the team, and embracing differences.

Responsibilities

• Evaluate quality: measure how low-precision computation changes results on coding and reasoning benchmarks, and identify where quantization hurts most.
• Research quantization: implement and compare post-training quantization methods that raise inference efficiency while keeping task accuracy.
• Build reproducible tooling: automate evaluations, track configurations and results, and make experiments usable by engineers and AI agents.
• Measure runtime: characterize latency, throughput and memory use to find bottlenecks in different deployment setups.
• A possible first project is mapping the quality versus performance Pareto frontier of a Qwen or LlaMa model, testing 16-bit, 8-bit and 4-bit quantization to find the best runtime and quality tradeoff.

Requirements

• You are currently enrolled in a master's program in Computer Science, Mathematics, Electrical Engineering, Machine Learning or a related field.
• You have strong Python and PyTorch skills, including reading and changing model implementations.
• You use AI coding tools effectively and review their output critically.
• You think like an experimenter: you form hypotheses, design fair comparisons and draw careful conclusions from noisy results.
• You communicate clearly and can present experimental results and technical tradeoffs to the team.
• You take on responsibility, learn quickly and act on your own initiative.
• Strong pluses: experience with quantization, LLM evaluation, performance profiling or inference optimization.

Conditions

This is a working student position.

You can work from Sunnyvale, CA or Munich, Germany, or remotely from North America or Europe.

Benefits

• Comprehensive benefits, competitive compensation, flexible spending options and recognition programs.
• Close mentorship from experienced engineers and ownership of a focused project.
• Hands-on work with leading open-source LLMs, building skills in quantization, inference optimization, experimental design, software engineering and AI coding tools.