The team builds datasets, evaluation rubrics, and infrastructure that help make advanced AI models better. You will work alongside research teams to run experiments, find the ways models fail, and convert training goals into working data and evaluation systems.
Responsibilities
Design data slices that expose meaningful model failures in finance, code, and enterprise workflows.
Build and improve evaluation rubrics and reward signals for RLHF and RLVR pipelines.
Study how annotators behave and run experiments aimed at stronger model capabilities.
Create quantitative measures of dataset quality, diversity, and effect on alignment and capabilities.
Set up and maintain real-world and synthetic data pipelines.
Work with AI research teams to turn training goals into concrete data and evaluation specs.
Requirements
You have 1 to 4 years of software engineering experience building reliable data pipelines, experiment infrastructure, or evaluation systems.
You write strong Python and have shipped software and infrastructure.
You have hands-on work with LLM evaluations, post-training, reward modeling, RLHF, RLVR, or related ML infrastructure.
You have created or assessed datasets, judging quality, diversity, coverage, and downstream model performance.
You reason well about experiments: you form hypotheses, define metrics, design controlled tests, and read noisy results.
You have designed data collection strategies and evaluation rubrics for machine learning models.