You and the process

With a person
Anna, in-house recruiter
9 years hiring engineers
30 minutes with a real recruiter
They read your CV with you, on a call, and say where the offers are being lost.
Didn’t find what you were looking for? Tell us what to build
Early Window: Closes in
Company hidden

Member of Technical Staff

  • Hybrid
  • 6+ years

Salary

$150,000 - 280,000/ year

AI summary

For members

The whole posting in a few lines. Sign up to read it here and on every role you open.

Sign Up to Read

Description

The company is at a turning point: frontier labs are racing to build general-purpose robots, and high-quality training data is the main constraint. It is building the base layer under foundation models, including data collection systems, operational capacity, an exabyte-scale data warehouse, and a model evaluation platform, for its partners.

The role is for an Embodied Capabilities researcher who works out what frontier models can do in the physical world and where they fail. Work includes designing the tasks behind benchmarks and RL environments, validating them on real robots, and using collected data to decide what to build next. Example projects include a benchmark suite that separates the strongest VLAs, locating tasks where LLM agents break down, and turning data findings into new tasks and targeted data collections.

Responsibilities

  • Own tasks from start to finish: what they measure, how scoring works, and whether the measurement is sound.
  • Design tasks for LLMs, VLAs, and WAMs that test specific robotics skills, from precise contact to long-horizon planning and recovering from errors.
  • Implement tasks on the evaluation platform, including reward functions, verifiers, success criteria, and environment configurations.
  • Check tasks on real hardware with operators to confirm they are feasible, guard against reward hacking, and set their difficulty.
  • Build simulated versions of tasks for RL environments, measuring real-world variation such as the spread of starting positions.
  • Maintain the task taxonomy that defines which capabilities each task tests and which environment it belongs to.
  • Analyze evaluation results and collected data to find capability gaps, unreliable tasks, and coverage holes.
  • Publish findings on where frontier models stand in the physical world.

Requirements

  • Strong Python programming skills.
  • Hands-on experience with robot manipulation and a sense of what makes a physical task difficult.
  • Strong data analysis skills, including work with large episode datasets and video.
  • Solid grounding in experimental design and statistics, and careful thinking about what a score really measures.
  • Working knowledge of current VLAs and LLM agents, including their strengths and limits.
  • A creative, adversarial mindset: you like designing challenges and finding ways around them.
  • Nice to have: experience designing robotics competitions or benchmarks, such as RoboCup, BEHAVIOR, RoboArena, or IROS and ICRA challenges.
  • Nice to have: experience with LLM-as-judge, reward modeling, or RL post-training.
  • Nice to have: experience with simulation such as MuJoCo, Isaac Sim, or similar, or with real2sim pipelines.
  • Nice to have: experience fine-tuning a VLA or running model baselines yourself.
  • Nice to have: experience with CAD or 3D printing for fixtures and repeatable setups.

Where you’d work

Part of the week in the office

You can work from

  • United States

About the company

Company hidden

  • Industry: AI

Office in United States

Your chances

Still hiring, not crowded yet, and you'd be among the first.

  • 17 checks run
  • 6 good signs
  • 0 red flags

Still hiring?

11 checks

Actively hiring

In its favour4

  • Still on the company's own careers site, checked 1 h agoModerate evidence
  • Found in the last 48 hours, before the big job boardsModerate evidence
  • Specific about the basics: pay, place, level, stack and contract all statedSlight evidence
1 moreFewer
  • States its salarySlight evidence

How crowded?

6 checks

Low

In its favour2

  • In its Early Window: not on the big job boards yetStrong evidence
  • Senior level: far fewer people qualifySlight evidence

Fits Me

How well does this role fit you?

Answer a few questions or drop your CV, and every role gets a fit score with the reasons, this one first.

  • Your field
  • Level
  • Stack
  • Work model
  • Salary floor
  • Must-haves
Details13 facts · Role, Location, Compensation, Employment, Company
Tech stack
  • Python
  • LLMs
Seniority
Staff
Type
Full-time
Industry
AI
Specialty
ML
Region
United States
Show 7 more factsShow less

Role

Category
Data & Analytics
Specialty
ML
Seniority
Staff
Experience
6+ years
Tech stack
  • Python
  • LLMs

Location

Work model
Hybrid
Region
United States
Office
  • United States
Remote from
  • United States

Compensation

Salary
$150,000 - 280,000 / year
Pay period
Annual

Employment

Type
Full-time

Company

Industry
AI

Something wrong with this vacancy?

Similar vacancies

Share this vacancy

What's wrong with it?

The employer never sees who reported.

Reason