You and the process

With a person
Anna, in-house recruiter
9 years hiring engineers
30 minutes with a real recruiter
They read your CV with you, on a call, and say where the offers are being lost.
Didn’t find what you were looking for? Tell us what to build

This vacancy is closed and no longer takes applications. See open Data & Analytics Jobs

Turing

Research Engineering Manager, STEM

  • Remote
  • 6+ years

Salary

Not stated

Description

The company builds large-scale datasets and reinforcement learning environments for frontier AI labs and enterprises. It is headquartered in San Francisco.

It works with frontier labs on high-quality datasets, RL environments and research benchmarks for software engineering, enterprise knowledge work and advanced STEM reasoning. It also deploys agentic AI systems inside critical workflows for Fortune 500 firms in financial services, life sciences, healthcare, retail, automotive and CPG.

The STEM Horizontal team produces the high-signal coding and STEM data that frontier labs train on. It is now building the research engineering behind its next layer: synthetic data that measurably improves models and benchmarks that catch what current evals miss.

The company plans to publish its findings, datasets and benchmarks openly.

Responsibilities

  • Manage a group of research engineers and work day to day with research scientists.
  • Recruit research engineers, selecting for research taste as well as engineering rigor.
  • Stay close enough to the code to review an eval harness line by line or prototype a generator when needed.
  • Run the team's operations: project selection, planning, experiment reviews, design docs and clear ownership.
  • Coach engineers on experiment design, code quality and presenting results. Give direct, regular feedback and run fair performance cycles.
  • Set the team's standards for eval methodology, data quality and reproducibility, so any dataset or result can be regenerated from a commit, a config and a seed.
  • Turn hypotheses into experiment plans with scientists, covering what is being tested, the baseline, the ablations and the result that would change the view. Bring your own hypotheses, not only implement theirs.
  • Move research prototypes to production scale without losing their scientific intent.
  • Own synthetic data pipelines that generate, verify, filter, deduplicate and version data, using programmatic generators, LLM-based generation, automated verifiers and targeted expert review.
  • Measure data by its effect on models through ablations and fine-tuning or RL experiments that show which attributes (difficulty, diversity, correctness, coverage) drive gains.
  • Build quality controls that hold up at volume: gold sets, LLM-judge calibration against expert labels, sampling audits and agreement metrics.
  • Find where frontier models fail and where benchmarks have saturated, then design evaluations for what is missing.
  • Own benchmark methodology end to end: task taxonomy, difficulty calibration, graders and verifiers, contamination controls, held-out splits and confidence intervals.
  • Ship deterministic, reproducible eval harnesses, including agentic and sandboxed-execution evals, that internal teams and partner labs trust.
  • Give the final sign-off on whether a result holds, since partner labs and internal teams act on it.
  • Publish benchmarks, datasets and findings with research scientists at venues such as NeurIPS, ICML and ICLR.
  • Write code every week on the hardest parts: core verifier logic, eval statistics and the pipeline components others depend on. Expect to spend a third or more of your time on direct technical work.
  • Be the person the team turns to for its hardest debugging and method questions, such as why a judge disagrees with experts or whether a gain is real.
  • Act as the technical counterpart to researchers at frontier labs: scope requests, set acceptance criteria and explain results and trade-offs plainly.
  • Work with strategic project leads, delivery and engineering teams so research reaches client work and the team's platforms.
  • Example work: design and ship a benchmark for a capability gap and see partner labs adopt it; build a long-horizon coding pipeline with execution-based verifiers and measure which task properties drive RL gains; calibrate an LLM judge against expert labels and decide where humans stay involved.

Requirements

  • 8+ years of hands-on ML or research engineering, at least 5 of them in industry, including 3+ years directly managing research engineers, ML engineers or researchers, covering hiring and performance management.
  • Has owned synthetic or curated data pipelines in production that drove training decisions, and can show how quality and downstream impact were measured.
  • Has designed and shipped LLM evals or benchmarks that others used for model decisions. Understands contamination, saturation, judge bias, inter-annotator agreement, variance and confidence intervals.
  • Plans ablations up front, chooses baselines honestly and can tell a real effect from noise.
  • Strong Python. Production experience with PyTorch or JAX. Working knowledge of SFT, DPO, PPO/GRPO and RL with verifiable rewards.
  • Has set direction for a team, not only carried out a roadmap handed over.
  • Has run distributed data processing and large evaluation or training jobs (for example Ray, Spark or Kubernetes) on AWS or GCP.
  • Writes clearly for scientists and non-specialists, and is comfortable presenting results to researchers at partner labs.
  • Uses LLM coding tools such as Claude Code, Cursor or Copilot daily, and reviews their output critically. This is a must-have.
  • PhD or Master's degree in AI, machine learning, computer science, statistics or a closely related field. PhD preferred. Exceptional equivalent research experience will also be considered, with industry outcomes weighted most heavily.
  • Strongly preferred: post-training data, evals or reward modeling experience at a frontier AI lab or a top industry research group.
  • Strongly preferred: has built or contributed to a public benchmark or dataset that the community uses.
  • Strongly preferred: first-author papers at NeurIPS, ICML, ICLR, ACL, EMNLP or similar venues.
  • Strongly preferred: RL environment and verifier design for agentic tasks such as coding agents, tool use, computer use or long-horizon workflows.
  • Strongly preferred: data valuation or attribution, measuring what each slice of data contributes to a model.
  • Strongly preferred: depth in a STEM domain such as math, physics, chemistry or biology. Not required.
  • Strongly preferred: has managed tech leads or managers, or run several concurrent programs.
  • Strongly preferred: has directed large expert networks through specs, rubrics and fast feedback loops.

Conditions

Applicants from the European Union should review the company's GDPR notice.

Benefits

  • Work on post-training, evaluation and synthetic data alongside leading AI labs and enterprises.
  • Build datasets and benchmarks that shape how frontier labs measure progress in coding and STEM reasoning.
  • High autonomy, rapid iteration and meaningful commercial impact.
  • Work alongside colleagues from Google, Meta, Amazon and other leading technology companies.
  • Opportunity to publish research at ICLR, ICML and NeurIPS.
  • Startup speed, with ownership and the chance to see your work have real impact.

Where you’d work

Fully remote, from anywhere

You can work from

  • Anywhere in the world

About the company

Turing

  • Industry: AI

Also hiring in Los Angeles, United States and New York, United States

1 of their 3 open roles is remote

Details11 facts · Role, Location, Compensation, Company
Tech stack
  • Python
  • PostgreSQL
  • AWS
  • Kubernetes
  • Docker
  • Terraform
  • PyTorch
  • LLMs
Seniority
Manager
Industry
AI
Specialty
ML
Region
United States
Pay period
Annual
Show 5 more factsShow less

Role

Category
Data & Analytics
Specialty
ML
Seniority
Manager
Experience
8+ years
Tech stack
  • Python
  • PostgreSQL
  • AWS
  • Kubernetes
  • Docker
  • Terraform
  • PyTorch
  • LLMs

Location

Work model
Remote
Region
United States
Remote from
  • Anywhere

Compensation

Salary
Salary by agreement
Pay period
Annual

Company

Industry
AI

Something wrong with this vacancy?

Similar vacancies

Share this vacancy

What's wrong with it?

The employer never sees who reported.

Reason