You and the process

With a person
Anna, in-house recruiter
9 years hiring engineers
30 minutes with a real recruiter
They read your CV with you, on a call, and say where the offers are being lost.
Didn’t find what you were looking for? Tell us what to build
xDOF

Member of Technical Staff, Evaluations

  • Hybrid
  • 6+ years

Salary

$180,000 - 300,000/ year

AI summary

For members

The whole posting in a few lines. Sign up to read it here and on every role you open.

Sign Up to Read

Description

The company is at a turning point: frontier labs are racing to build general-purpose robots, and high-quality training data is the main constraint. It is building the layer under foundation models, including data collection systems, operational capacity, an exabyte-scale data warehouse, and a model evaluation platform, to support its partners.

The team is hiring an Evaluations Software Engineer to build the methods the field uses to measure robot intelligence. You will create benchmarks that show partners how their LLMs, VLAs, and WAMs perform on physical tasks, both in simulation and on real hardware. Work may include designing benchmark tasks, building the simulation pipeline, checking how well simulation predicts real-world results, and bringing new research into the evaluation workflow.

Responsibilities

  • Own how models are measured and what the team learns from those measurements.
  • Design benchmark tasks for LLMs, VLAs, and WAMs that test real robotics skills, from long-horizon planning to dexterous manipulation.
  • Build verifiers, success detectors, and progress metrics that score episodes reliably, in simulation and on real hardware.
  • Build harnesses that let LLMs and coding agents control robots, either directly or by planning over VLAs.
  • Measure how well simulation results predict real-world performance, and work to improve that match.
  • Fine-tune open-source VLAs in a consistent way so every model gets a fair baseline.
  • Analyze failures across runs to locate capability gaps, then turn them into targeted data collection and RL environments.
  • Produce results for partners with rigorous statistics and clear reporting.

Requirements

  • Strong programming skills in Python or C++.
  • Experience building benchmarks, evaluation pipelines, or graders for ML models, with careful thought about what a score really measures.
  • Hands-on experience with robot learning, with simulation tools such as MuJoCo, Isaac Sim, or similar, or with deploying policies on real hardware.
  • Experience working with LLM or coding agents, and the ability to read a trace to work out why a model failed.
  • An adversarial mindset: when you see a metric, your first question is how it could be gamed.
  • Experience building and shipping software in fast-moving environments.

Where you’d work

Part of the week in the office

You can work from

  • United States

About the company

xDOF

  • Industry: AI

Office in United States

Your chances

Worth a look before you spend an evening tailoring a CV for it.

  • 21 checks run
  • 1 red flag

Still hiring?

14 checks

No red flags

How crowded?

7 checks

1 red flag

Fits Me

How well does this role fit you?

Answer a few questions or drop your CV, and every role gets a fit score with the reasons, this one first.

  • Your field
  • Level
  • Stack
  • Work model
  • Salary floor
  • Must-haves
Details12 facts · Role, Location, Compensation, Employment, Company
Tech stack
  • Python
  • C++
  • LLMs
Seniority
Staff
Type
Full-time
Industry
AI
Region
United States
Pay period
Annual
Show 6 more factsShow less

Role

Category
Development
Seniority
Staff
Experience
6+ years
Tech stack
  • Python
  • C++
  • LLMs

Location

Work model
Hybrid
Region
United States
Office
  • United States
Remote from
  • United States

Compensation

Salary
$180,000 - 300,000 / year
Pay period
Annual

Employment

Type
Full-time

Company

Industry
AI

Something wrong with this vacancy?

Similar vacancies

Share this vacancy

What's wrong with it?

The employer never sees who reported.

Reason