The company is at a turning point: frontier labs are racing to build general-purpose robots, and high-quality training data is the main constraint. It is building the base layer under foundation models, including data collection systems, operational capacity, an exabyte-scale data warehouse, and a model evaluation platform, for its partners.
The role is for an Embodied Capabilities researcher who works out what frontier models can do in the physical world and where they fail. Work includes designing the tasks behind benchmarks and RL environments, validating them on real robots, and using collected data to decide what to build next. Example projects include a benchmark suite that separates the strongest VLAs, locating tasks where LLM agents break down, and turning data findings into new tasks and targeted data collections.
Responsibilities
Own tasks from start to finish: what they measure, how scoring works, and whether the measurement is sound.
Design tasks for LLMs, VLAs, and WAMs that test specific robotics skills, from precise contact to long-horizon planning and recovering from errors.
Implement tasks on the evaluation platform, including reward functions, verifiers, success criteria, and environment configurations.
Check tasks on real hardware with operators to confirm they are feasible, guard against reward hacking, and set their difficulty.
Build simulated versions of tasks for RL environments, measuring real-world variation such as the spread of starting positions.
Maintain the task taxonomy that defines which capabilities each task tests and which environment it belongs to.
Analyze evaluation results and collected data to find capability gaps, unreliable tasks, and coverage holes.
Publish findings on where frontier models stand in the physical world.
Requirements
Strong Python programming skills.
Hands-on experience with robot manipulation and a sense of what makes a physical task difficult.
Strong data analysis skills, including work with large episode datasets and video.
Solid grounding in experimental design and statistics, and careful thinking about what a score really measures.
Working knowledge of current VLAs and LLM agents, including their strengths and limits.
A creative, adversarial mindset: you like designing challenges and finding ways around them.
Nice to have: experience designing robotics competitions or benchmarks, such as RoboCup, BEHAVIOR, RoboArena, or IROS and ICRA challenges.
Nice to have: experience with LLM-as-judge, reward modeling, or RL post-training.
Nice to have: experience with simulation such as MuJoCo, Isaac Sim, or similar, or with real2sim pipelines.
Nice to have: experience fine-tuning a VLA or running model baselines yourself.
Nice to have: experience with CAD or 3D printing for fixtures and repeatable setups.