Evaluation Engineer

Description

The company designs, builds and deploys intelligent robots for logistics and manufacturing, aiming to build the first general-purpose robotic system for industrial work.

The team is made up of engineers, researchers and operators with backgrounds in AI, robotics, engineering and autonomous systems. The work sits between applied research and real-world engineering, taking ideas through to prototypes and deployed systems.

The role is a Robotics Evaluation Engineer. You will build the systems that measure, diagnose and improve robot policy performance in simulation and on real robots. You will own the core evaluation infrastructure that lets the AI team compare models, understand failure modes, measure generalization and reliability, and decide on data, training and deployment.

You will work with robot learning researchers and engineers across simulation, perception, controls, data and infrastructure to define task success, build reproducible evaluation frameworks, automate testing and build tools that explain why policies succeed or fail.

Responsibilities

• Own the design, build and ongoing development of the robot learning evaluation framework across simulation and real robots.
• Set task specifications, success criteria, failure categories, metrics and benchmark protocols for manipulation policies.
• Build automated tools that compare models, catch regressions, run large evaluations at scale and produce clear performance reports.
• Create simulation environments, task scenarios and procedural variations for policy evaluation.
• Build repeatable real-robot evaluation workflows that need little manual supervision and yield statistically sound results.
• Track and improve how well simulated performance predicts real-world performance.
• Build tools to replay, visualize and diagnose policy failures in simulation and on physical robots.
• Work with robot learning researchers to find whether a failure comes from the model, data, perception, controls, calibration, simulation, hardware or long-horizon execution.

Requirements

• Significant experience building systems for robotics evaluation, simulation, testing or benchmarking.
• Strong programming skills in Python, C++ or a similar robotics or systems language.
• Experience with robotics simulators such as Isaac Sim, MuJoCo, Gazebo or similar.
• Experience evaluating learned robot policies in simulation and on physical robots.
• Experience designing reproducible experiments, metrics, benchmarks and failure analysis workflows.
• Ability to build reliable software for automated testing, data collection, visualization and reporting.
• Strong grasp of robotic systems and the skill to debug problems across software, models, sensors, controls and hardware.
• A track record of owning complex technical projects from initial design through deployment and ongoing operation.
• Familiarity with sim-to-real transfer, domain randomization, sensor modeling and system identification.
• Experience importing and configuring simulation assets, including robot models, objects, joints, collisions, inertial properties, materials and sensors.
• Experience with hardware-in-the-loop testing, continuous integration for robotics or automated robot test cells.
• Familiarity with multimodal robot policies, vision-language-action models or other learned manipulation architectures.
• Strong background in statistical analysis, confidence estimation and experimental design.