You and the process

With a person
Anna, in-house recruiter
9 years hiring engineers
30 minutes with a real recruiter
They read your CV with you, on a call, and say where the offers are being lost.
Didn’t find what you were looking for? Tell us what to build
NVIDIA

Inference Performance Engineer, Agent Driven Inference Optimization

  • Hybrid
  • 3-6 years

Salary

$124,000 - 195,500/ year

AI summary

For members

The whole posting in a few lines. Sign up to read it here and on every role you open.

Sign Up to Read

Description

The company is hiring a senior inference performance engineer to raise the limits of its performance on large-scale AI inference benchmarks.

You will work in an autonomous optimization framework where AI agents repeatedly run benchmarks, profile workloads and tune configurations, so each technique you design is applied many times over.

This suits someone who enjoys getting the most out of GPUs and wants their work to reach beyond what they can do by hand.

Responsibilities

  • Turn your performance know-how into reusable skills, workflows and evidence-based methods that AI agents can run on their own.
  • Review experiments that agents generate, check their findings and maintain the best-known configurations.
  • Improve throughput per GPU and user interactivity for AI inference workloads by testing configuration options, parallelism, batching, KV cache handling, quantization and speculative decoding settings.
  • Measure and tune both aggregated and disaggregated serving setups across TensorRT-LLM, SGLang, vLLM and Dynamo on the company's newest GPU platforms.
  • Profile workloads with Nsight Systems, kernel traces and internal analysis tools, use roofline and speed-of-light analysis to locate realistic headroom, and carry fixes from hypothesis through to measured gains.
  • Contribute improvements upstream, including serving framework patches, optimized kernels and deployment recipes that push the public Pareto frontier while keeping model correctness strict.
  • Work with the TensorRT-LLM, SGLang, vLLM, kernel, benchmarking and GPU architecture teams to turn profiling findings into shipped performance gains.

Requirements

  • Hold a BS, MS or PhD in computer science, computer engineering, electrical engineering, applied math or a related field, or have equivalent experience.
  • Have 3+ years of relevant engineering experience.
  • You must have deep knowledge of how AI models run efficiently, including continuous batching, throughput-latency tradeoffs, KV cache and memory limits, parallel processing, MoE serving, quantization and meeting serving SLAs.
  • You must have hands-on experience benchmarking and profiling GPU workloads with tools such as Nsight Systems, Nsight Compute, CUPTI or the PyTorch profiler, and reading kernel-level performance data.
  • Write strong Python and be able to work through and change large C++/CUDA serving codebases.
  • Run disciplined experiments that change one variable at a time, produce reproducible benchmarks and base optimization decisions on evidence.
  • Communicate clearly in writing and speaking, explaining performance tradeoffs to people and writing documentation that autonomous systems can use.
  • Stand out with direct contributions to TensorRT-LLM, vLLM, SGLang, FlashInfer, Dynamo or comparable inference frameworks.
  • Stand out with experience in disaggregated serving, wide expert-parallel MoE inference, KV cache transfer, or NCCL, NIXL or NVSHMEM communication across multiple nodes.
  • Stand out with CUDA kernel work or optimization on Hopper or Blackwell, focused on Tensor Cores, TMA and warp specialization.
  • Stand out with proven results on public inference benchmarks such as MLPerf Inference or SemiAnalysis InferenceX.
  • Stand out with experience building or running agentic AI workflows that automate engineering tasks.

Conditions

This is a hybrid role based in Level 2 or Level 3 positions, with base pay set by location, experience and other objective job factors.

Base salary ranges are 124,000 USD to 195,500 USD for Level 2, and 152,000 USD to 241,500 USD for Level 3.

This is an existing vacancy. Applications are accepted at least until November 13, 2026.

The company uses AI tools in its recruiting process.

The company is an equal opportunity employer and does not discriminate on any legally protected basis.

Benefits

  • The company describes its salaries and benefits as highly competitive and comprehensive, with details of family benefits available on its benefits site.

Where you’d work

Part of the week in the office

You can work from

  • United States

About the company

NVIDIA

Also hiring in United States, Redmond, United States, Seattle, United States and 15 more places

161 of their 480 open roles are remote

Your chances

Worth a look before you spend an evening tailoring a CV for it.

  • 21 checks run
  • 1 red flag

Still hiring?

14 checks

1 red flag

How crowded?

7 checks

No red flags

Fits Me

How well does this role fit you?

Answer a few questions or drop your CV, and every role gets a fit score with the reasons, this one first.

  • Your field
  • Level
  • Stack
  • Work model
  • Salary floor
  • Must-haves
Details11 facts · Role, Location, Compensation, Employment
Tech stack
  • Python
  • PyTorch
  • C++
  • LLMs
  • CUDA
Type
Full-time
Equity
Equity offered
Specialty
Backend
Region
United States
Pay period
Annual
Show 5 more factsShow less

Role

Category
Development
Specialty
Backend
Experience
3+ years
Tech stack
  • Python
  • PyTorch
  • C++
  • LLMs
  • CUDA

Location

Work model
Hybrid
Region
United States
Remote from
  • United States

Compensation

Salary
$124,000 - 195,500 / year
Pay period
Annual
Equity
Equity offered

Employment

Type
Full-time

Something wrong with this vacancy?

Similar vacancies

Share this vacancy

What's wrong with it?

The employer never sees who reported.

Reason