You and the process

With a person
Anna, in-house recruiter
9 years hiring engineers
30 minutes with a real recruiter
They read your CV with you, on a call, and say where the offers are being lost.
Didn’t find what you were looking for? Tell us what to build

This vacancy is closed and no longer takes applications. See open Data & Analytics Jobs

Turing

Research Engineering Manager, STEM

  • Office
  • 6+ years

Salary

$260,000 - 350,000/ year

Description

The company is based in San Francisco and works with frontier AI labs to produce high-quality datasets, reinforcement learning environments, and research benchmarks that improve model skills in software engineering, enterprise knowledge work, and advanced STEM reasoning.

In software engineering, the company is the largest and longest-running data provider in its category.

It also works with Fortune 500 companies in financial services, life sciences, healthcare, retail, automotive, and consumer packaged goods, building and deploying end-to-end agentic AI systems inside critical workflows.

Working on both sides lets the company feed deployment results back into better data, evaluations, and models.

The STEM Horizontal team creates the high-signal coding and STEM data that frontier labs train on. This role leads the research engineering for its next stage: synthetic data that measurably improves models, and new benchmarks that catch what current evals miss.

Responsibilities

  • Lead a team of research engineers while working closely with research scientists.
  • Stay close to the code: review evaluation harnesses line by line and prototype generators yourself when needed.
  • Hire and grow research engineers, selecting for research judgment as well as engineering rigor. Hiring is a central part of the job.
  • Ideally you have worked at a frontier lab and know what makes a lab trust a dataset or benchmark result.
  • Build synthetic data pipelines that generate, verify, and filter every sample, and show which data properties drive downstream gains, not only volume.
  • Create benchmarks for capabilities that current tests miss, with calibrated difficulty, contamination controls, and statistics strong enough to support a decision.
  • Catch quiet quality failures early, such as a subtle verifier bug that ships wrong labels into a training run and wastes weeks.
  • Define hiring profiles and run the team's planning, project selection, experiment reviews, design docs, and clear ownership.
  • Coach engineers on experiment design, code quality, and presenting results, with regular direct feedback and fair performance reviews.
  • Set team standards for evaluation method, data quality, and reproducibility, so every dataset and result can be regenerated from a commit, config, and seed.
  • Turn hypotheses into experiment plans with research scientists: the test, the baseline, the ablations, and the outcome that would change your mind. Bring your own hypotheses, not only theirs.
  • Move research prototypes to production scale without losing their scientific intent.
  • Own pipelines that generate, verify, filter, deduplicate, and version data, mixing programmatic generators, LLM generation, automated verifiers, and targeted expert review.
  • Measure data by its effect on models through ablations and fine-tuning or RL experiments.
  • Build quality controls that hold up at volume: gold sets, LLM judge calibration against expert labels, sampling audits, and agreement metrics.
  • Find where frontier models fail and where benchmarks have saturated, then design evaluations for what is missing.
  • Own benchmark method from start to finish: task taxonomy, difficulty calibration, graders, contamination controls, held-out splits, and confidence intervals.
  • Ship deterministic, reproducible eval harnesses, including agentic and sandboxed execution evals, that internal teams and partner labs trust.
  • Sign off on whether a result holds, since partner labs and internal teams act on your decision.
  • Publish benchmarks, datasets, and findings with research scientists at venues such as NeurIPS, ICML, and ICLR.
  • Write code every week on the hardest parts: core verifier logic, eval statistics, and foundational pipeline components. Spend a third or more of your time on direct technical work.
  • Serve as the team's go-to person for hard debugging and method questions, such as why a judge disagrees with experts or whether a gain is real.
  • Act as the technical contact for researchers at frontier labs: scope requests, set acceptance criteria, and explain results and trade-offs clearly.
  • Work with strategic project leads, delivery, and engineering so research reaches client work and team platforms.
  • Example projects: design and ship a benchmark for a capability gap that partner labs adopt; build a long-horizon coding synthetic pipeline with execution-based verifiers and ablate which task traits drive RL gains; calibrate an LLM judge against experts and decide where humans stay involved; design the experiment that shows whether a 2-point gain on a partner's eval is real; choose among three synthetic data approaches through experiments.
  • Tech stack: Python, PyTorch or JAX, SFT, DPO, PPO/GRPO, RL with verifiable rewards, LLM-as-judge, programmatic verifiers, sandboxed execution, gold sets, contamination checks, Django or FastAPI, PostgreSQL, Docker, Terraform, AWS or GCP.

Requirements

  • You have 8+ years of hands-on ML or research engineering, at least 5 in industry, including 3+ years directly managing research engineers, ML engineers, or researchers, with hiring and performance management included.
  • You have owned production synthetic or curated data pipelines that drove training decisions, and can show how you measured their quality and downstream impact.
  • You have designed and shipped LLM evals or benchmarks that others used for model decisions, and you understand contamination, saturation, judge bias, inter-annotator agreement, variance, and confidence intervals.
  • You plan ablations up front, pick baselines honestly, and separate real effects from noise.
  • You write strong Python, have production experience with PyTorch or JAX, and know the post-training stack: SFT, DPO, PPO/GRPO, and RL with verifiable rewards.
  • You have set direction for a team rather than only running a roadmap someone handed you.
  • You have run distributed data processing and large evaluation or training jobs with tools like Ray, Spark, or Kubernetes on AWS or GCP.
  • You write clearly for scientists and non-specialists, and you are comfortable presenting results to researchers at partner labs.
  • You use LLM coding tools such as Claude Code, Cursor, or Copilot daily and review their output critically. This is a must-have.
  • You hold a PhD or Master's in AI, machine learning, computer science, statistics, or a closely related field. PhD preferred; exceptional equivalent research experience is considered, and industry outcomes weigh most.
  • Strongly preferred: post-training data, evals, or reward modeling work at a frontier lab or top industry research group, such as OpenAI, Anthropic, Google DeepMind, Meta, or xAI.
  • Strongly preferred: you built or contributed to a public benchmark or dataset the community uses.
  • Strongly preferred: first-author papers at NeurIPS, ICML, ICLR, ACL, EMNLP, or similar venues.
  • Strongly preferred: RL environment and verifier design for agentic tasks such as coding agents, tool use, computer use, or long-horizon workflows.
  • Strongly preferred: data valuation or attribution, measuring what each slice of data contributes to a model.
  • Strongly preferred: depth in a STEM field such as math, physics, chemistry, or biology. Not required.
  • Strongly preferred: experience managing tech leads or managers, or running several programs at once.
  • Strongly preferred: experience directing large expert networks through specs, rubrics, and fast feedback loops.

Conditions

Pay is $260,000 to $350,000 plus equity.

You do not need to meet every requirement to apply.

Applicants from the European Union should review the company's GDPR notice.

Benefits

  • Work directly with leading AI labs and enterprises on post-training, evaluation, and synthetic data.
  • Build the datasets and benchmarks that improve and measure advanced AI systems, and help shape how labs track progress in coding and STEM reasoning.
  • High autonomy, fast iteration, and real commercial impact.
  • Work alongside colleagues from Google, Meta, Amazon, and other major technology companies.
  • Opportunities to publish research at ICLR, ICML, and NeurIPS.
  • Bring frontier research into enterprise work by applying lessons from leading labs to business problems.
  • Startup pace with startup-level ownership and impact.

Where you’d work

From the office

About the company

Turing

  • Industry: AI

Office in United States

Also hiring in Los Angeles, United States and New York, United States

2 of their 5 open roles are remote

Details12 facts · Role, Location, Compensation, Company
Tech stack
  • Python
  • PostgreSQL
  • AWS
  • Kubernetes
  • Docker
  • Terraform
  • PyTorch
  • LLMs
Seniority
Manager
Equity
Equity offered
Industry
AI
Specialty
ML
Region
United States
Show 6 more factsShow less

Role

Category
Data & Analytics
Specialty
ML
Seniority
Manager
Experience
8+ years
Tech stack
  • Python
  • PostgreSQL
  • AWS
  • Kubernetes
  • Docker
  • Terraform
  • PyTorch
  • LLMs

Location

Work model
Office
Region
United States
Office
  • United States

Compensation

Salary
$260,000 - 350,000 / year
Pay period
Annual
Equity
Equity offered

Company

Industry
AI

Something wrong with this vacancy?

Similar vacancies

Share this vacancy

What's wrong with it?

The employer never sees who reported.

Reason