Senior Inference Optimization ML Engineer

Description

The company is building generalist intelligent robots. It owns the full robotics stack, including high-performance hardware, robot systems, infrastructure, and the foundation world models that control the robots.

The robots are meant to work in complex real-world settings and handle rare edge cases, based on the company's research and end-to-end system design.

The company has raised over $450M and is expanding model research, infrastructure, hardware development, and manufacturing scale-up.

Serving large multimodal foundation models for a growing robot fleet is costly, and the company does not want to depend on one hardware vendor or the priciest accelerators. This role makes those models run fast, correct, and cheap across AMD Instinct GPUs, Google TPUs, wafer-scale and dataflow accelerators, consumer GPUs such as the RTX 5090, and the Jetson/Thor-class compute on the robots.

You will choose which chip handles which workload and then make it perform well there. You will work daily with the research, robotics, and infrastructure teams.

Responsibilities

• Own inference from end to end, reducing latency, raising throughput, and lowering cost per inference for the vision, video, language, proprioception, and action models.
• Port and tune models for AMD (ROCm, HIP, MIOpen/hipBLASLt, Composable Kernel), TPU (XLA, JAX, Pallas, PyTorch/XLA), wafer-scale and dataflow accelerators through their SDKs and compilers, and consumer Blackwell GPUs like the RTX 5090, which offer FP8/FP4, limited memory, and multi-GPU over PCIe.
• Deploy and optimize models on Jetson/Thor-class embedded hardware within tight latency, power, thermal, and memory limits, and decide how work is divided between the robot and the cloud to meet real-time control needs.
• Measure hardware with rigor by building cost and performance models (perf/$, perf/W, memory capacity and bandwidth, interconnect) for each chip and model family, then recommend which hardware to buy or rent for serving, offline evaluation, and on-robot use.
• Write kernels that run across vendors where practical (Triton, Pallas, MLIR-based compilers), and vendor-specific kernels when the speedup justifies them. Handle attention, KV cache, memory layout, and fused operators on every backend.
• Apply compression so models fit and run well on less memory or limited data-type support, using quantization (INT8, FP8, FP4/MX, AWQ/GPTQ), pruning, distillation, and model or tensor parallelism.
• Keep outputs consistent across chips by writing tests for numerical parity, policy quality, and robot-relevant behaviour.
• Build benchmarking and regression tracking across hardware, covering latency baselines, throughput curves, and cost per inference for each model version on each target, with automatic regression alerts.
• Work with research engineers to design models that stay efficient and portable from the start, such as by avoiding operators that exist on only one vendor's hardware.

Requirements

• 2+ years of experience in inference optimization, ML systems, ML compilers, or a closely related field.
• Deep hands-on experience with PyTorch.
• A strong grasp of how compute, memory bandwidth, memory capacity, and interconnect limits shift between hardware architectures.
• Experience with model optimization such as quantization (INT8/FP8/FP4, AWQ), distillation, pruning, and compilers (torch.compile, XLA, TensorRT, IREE/MLIR, TVM, or vendor compilers).
• Familiarity with serving frameworks and their hardware support, such as vLLM on ROCm/TPU, SGLang, JetStream, and Triton Inference Server.
• Strong debugging and measurement skills. You turn vague complaints like "slow on chip X" or "too expensive" into specific bottlenecks, experiments, and verified gains.
• High ownership and a pragmatic approach. You weigh the total cost of a solution, not only peak benchmark numbers.
• Experience optimizing inference for diffusion models.
• Production or in-depth project experience on at least one accelerator stack other than NVIDIA datacenter GPUs, such as AMD ROCm, Google TPU, wafer-scale or dataflow accelerators, Groq, Tenstorrent, AWS Trainium/Inferentia, Intel Gaudi, or similar, or serious work deploying on consumer GPUs.
• Kernel authoring on more than one backend, such as HIP or Composable Kernel on AMD, Pallas on TPU, Triton on NVIDIA and AMD, or CUTLASS/CuTe.
• Contributions to open-source ML compilers or hardware backends such as PyTorch, OpenXLA, Triton, vLLM, IREE, or TVM.
• Experience running inference on wafer-scale systems, TPU pods, or fleets of consumer GPUs.
• Hands-on work with Jetson/Thor-class platforms, including TensorRT on Jetson, JetPack, DLA, and power and thermal profiling.
• Experience with multimodal or video model inference, including variable-length sequences and packing or bucketing.
• Experience with speculative decoding, continuous batching, disaggregated prefill/decode, or other LLM serving optimizations.
• Experience with streaming or low-latency systems for real-time robot control.
• Experience working with hardware vendors on driver or compiler bugs and early-access silicon.