Systems Performance Modeling Engineer

Description

The company builds high-performance, low-power generative AI inference systems, consisting of custom silicon, hardware and software for hyperscaler and neocloud data center customers.

The team is based in Sunnyvale, CA and Munich, Germany, with additional members working remotely across North America and Europe.

The company is a well funded startup that moves quickly.

The role is to build the models and tools that predict how generative AI inference workloads perform on the company systems, from a single accelerator up to rack, pod, and cluster scale.

This is a hands-on role for an engineer who enjoys writing simulator code, running experiments, and finding out why a prediction and a measurement disagree.

You will work with architects and the silicon, hardware, networking, and software teams to capture workload behavior, extend simulation and analytical models of the silicon, interconnect, and multi-hop fabrics, and check them against real hardware.

You will need to move across the stack, from the model graph through collectives to the network fabric, to locate where performance is lost.

Responsibilities

• Build and extend simulation-based performance models for multimodal generative AI inference at rack, pod, and cluster scale, covering compute, memory, collective communication, and network fabric.
• Model how serving strategies, including tensor, pipeline, and expert parallelism, prefill/decode disaggregation, batching, and KV-cache placement, interact with the company silicon and fabric topology, and measure their effect on latency, throughput, and cost per token.
• Build trace-capture and replay tools that record real execution from the inference runtime and replay it under hypothetical silicon, system, and network configurations.
• Model collective communication on multi-hop scale-out fabrics, including writing custom collective algorithms for the company's topology.
• Run calibration experiments on the company hardware as systems come online, compare results with model predictions, and correct the sources of error.
• Run design-space sweeps and write clear analyses that architects and engineering teams use to decide ASIC, fabric, and system configurations.
• Create performance projections for product and customer discussions.
• Keep the modeling codebase fast, tested, and reproducible so other engineers can run it on their own.

Requirements

• Hands-on experience building performance models, simulators, or analytical tools for ML workloads, distributed systems, or computer architecture.
• Solid understanding of distributed ML execution, including parallelism strategies and collective communication such as All-Reduce, All-Gather, and All-to-All, and how they scale.
• Working knowledge of system architecture across compute, memory, interconnect, and networking, with the ability to reason about bottlenecks between them.
• Experience comparing model predictions with real measurements and debugging where they diverge.
• Strong C++ and Python programming skills, with clean, testable, maintainable code.
• Ability to turn a loosely defined performance question into experiments and deliver results with little guidance.
• Clear written and verbal communication, especially when presenting data and trade-offs to other engineers.
• MS or higher in Computer Science, Computer Engineering, Electrical Engineering, or a related field.
• Familiarity with LLM inference serving, including batching, KV-cache management, disaggregated prefill/decode, and latency/throughput trade-offs.
• Experience modeling or benchmarking collective communication libraries such as NCCL, RCCL, or similar on real clusters.
• Background in data center or HPC networking, including topologies, RDMA/RoCE, and congestion behavior.
• Experience profiling ML workloads on accelerators such as GPUs, TPUs, or custom ASICs.
• Exposure to hardware/software co-design or early-stage architecture evaluation.
• Publications or open-source contributions in ML systems, architecture, or networking.
• Values the company holds: put people first; ethics and integrity always, being open, honest, and respectful; think big with ambitious goals of global scale; aim for excellence in everything; own it and get it done, since results matter; make each person better together than alone; embrace differences between people.

Conditions

The company is an equal-opportunity employer.

Benefits

• Comprehensive benefits, competitive compensation, flexible spending options, and recognition programs.