Description
About the company
The company is a frontier data research lab. We build the evals and environments that the world's leading frontier labs use to measure and train their models.
At the same time, we've expanded into other domains besides coding: RSI & Auto-Research, Law, Enterprise Business, Cybersecurity, and others. Now, we are facing more lab demand for our data than we can serve, and are rapidly scaling the team to grow the business.
Our approach to creating training data scales using technology, and all of our data products are built on a unified self-reinforcing platform that learns through experience.
You would be joining a close-knit team that has reached product market fit, and your work would directly help to multiply our revenue.
You can see some of our work here: the company's site (the company's site)
About the role
As a Research Scientist at the company, you'll own measuring and improving how models learn from our tasks. The job is to maximize learning signal we produce per unit time. You'll draw on your own experience and collaborate with researchers at the frontier to validate our data quality, identify where improvements are needed, and create new datasets. To succeed, you'll need to have extensive experience doing this work in production at a frontier lab.
Examples of what you’ll do
Work with our customers — researchers at frontier labs — to design novel post-training recipes and data quality measurement techniques
Design and run our in-house post-training stack to measure model lift on our tasks
Develop new data products based on datasets and experts available to us
Identify opportunities to take advantage of self-reinforcing exponential feedback loops
Create agents to analyze thousands of environments and millions of trajectories
Help curate and specify task distributions for new corpora
Work with procurement to ensure external data we acquire is suitable for refinement
Create scaleable systems for ingesting & evaluating data we are considering buying
Develop new techniques for mining data for signal