You and the process

With a person
Anna, in-house recruiter
9 years hiring engineers
30 minutes with a real recruiter
They read your CV with you, on a call, and say where the offers are being lost.
Didn’t find what you were looking for? Tell us what to build
Prime Intellect

Research Engineer - Data and Evaluation

  • Hybrid

Salary

$150,000 - 350,000/ year

AI summary

For members

The whole posting in a few lines. Sign up to read it here and on every role you open.

Sign Up to Read

Description

The company is building an open AI stack: the infrastructure that frontier AI labs usually keep in-house, offered to AI teams beyond those labs.

The platform covers compute, environments, evaluations, secure sandboxes, large-scale training, and deployment for post-training, including SFT, RL, tool use, and agent workflows.

The team builds open models trained end to end on long-horizon tasks such as autonomous research, along with the platform its own research staff uses.

The company has raised $150M in total funding from Founders Fund, Radical Ventures, NVIDIA, and a group of individual investors from AI, infrastructure, and enterprise software.

You will work alongside researchers to build the tasks and environments used to train the company's models with RL, and the evaluations that measure what those models can do, including investigating failures from task setup through to scoring.

Responsibilities

  • Create hard evaluations for coding and long-horizon agents.
  • Assemble runnable environments with realistic task setups and clear success criteria.
  • Write and test scoring logic and verifiers, looking for reward hacks and measures that give a false sense of progress.
  • Follow failures through model outputs and harness code to determine whether the cause is the model, the environment, or the scoring.
  • Build and scale RL data pipelines that produce and select difficult coding and agentic tasks.
  • Partner with RL researchers to check whether changes to tasks and training data improve results on held-out tasks.

Requirements

  • Strong Python software engineering, including building data pipelines, evaluation systems, or ML workflows that other people rely on.
  • Hands-on work with language models or agents, and an understanding of how training data and evaluation choices shape model behavior.
  • Skill at examining complex data and model outputs, finding failure modes, and turning a vague quality question into something you can test.
  • A track record of leading technical projects from experiments through implementation and validation, working closely with researchers and engineers.
  • Hands-on experience building technical evaluations or agent environments, with enough engineering depth to read model traces and debug harness and scoring code.
  • Relevant experience may include:
  • Large-scale synthetic data generation, dataset curation, or distributed processing for model training.
  • Evaluation design, executable coding tasks, agent environments, verifiers, or reward design.
  • Experiment analysis, statistical testing, contamination detection, or reproducible research tooling.
  • RL or post-training data work, such as choosing task difficulty or analyzing rollout and reward distributions.
  • Contributions to open-source RL or evaluation frameworks comparable to the ones the team uses.
  • Work on model-based grading, long-horizon coding agents, or diagnosing reward exploitation.
  • Depth in one area is enough: data generation, environments and rewards, or evaluation systems.

Conditions

Work remotely or in person at the San Francisco office, with flexible arrangements.

Benefits

  • Cash pay of $150-350k, plus equity incentives tied to the company's growth.
  • Full medical, dental, and vision coverage for employees and dependents.
  • Flexible PTO.
  • $200/month health and wellness benefit.
  • Visa sponsorship and relocation help for international candidates.
  • Budget for professional development, such as courses and conferences.
  • Quarterly team off-sites, hackathons, conferences, and learning events.
  • Meals covered by the company when you work from the San Francisco office.

Where you’d work

Part of the week in the office

You can work from

  • United States

Relocation offered

Visa sponsored

About the company

Prime Intellect

  • Industry: AI

Also hiring in San Francisco, United States

2 of their 11 open roles are remote

Your chances

Worth a look before you spend an evening tailoring a CV for it.

  • 23 checks run
  • 6 red flags

Still hiring?

14 checks

2 red flags

How crowded?

9 checks

4 red flags

Fits Me

How well does this role fit you?

Answer a few questions or drop your CV, and every role gets a fit score with the reasons, this one first.

  • Your field
  • Level
  • Stack
  • Work model
  • Salary floor
  • Must-haves
Details13 facts · Role, Location, Compensation, Employment, Company
Tech stack
  • Python
  • Databricks
  • LangChain
Type
Full-time
Equity
Equity offered
Industry
AI
Specialty
ML
Region
United States
Show 7 more factsShow less

Role

Category
Data & Analytics
Specialty
ML
Tech stack
  • Python
  • Databricks
  • LangChain

Location

Work model
Hybrid
Region
United States
Remote from
  • United States
Relocation
Offered
Visa sponsorship
Sponsored

Compensation

Salary
$150,000 - 350,000 / year
Pay period
Annual
Equity
Equity offered

Employment

Type
Full-time

Company

Industry
AI

Something wrong with this vacancy?

Similar vacancies

Share this vacancy

What's wrong with it?

The employer never sees who reported.

Reason