Research Engineer / Scientist, Pre-training

Description

The company builds reliable, interpretable and steerable AI systems meant to be safe and beneficial for users and for society.

You would join the Pre-training team, which develops the next generation of large language models, as a Research Engineer.

The team is small in its research focus but growing quickly. It works as one cohesive group on a few large-scale efforts, favouring impact over narrow problems.

Research here is treated as an empirical science, closer to physics and biology than to traditional computer science, and the team holds frequent research discussions.

Sample projects include optimizing the throughput of new attention mechanisms, comparing compute efficiency across Transformer variants, preparing large datasets for efficient model use, scaling distributed training to thousands of GPUs, designing fault tolerance for training infrastructure, and building interactive visualizations of model internals such as attention patterns.

Responsibilities

• Design, run and analyze scientific experiments to deepen the team's understanding of large language models.
• Do research and build solutions in model architecture, algorithms, data processing and optimizer development.
• Lead small research projects on your own and contribute to larger initiatives with the team.
• Optimize and scale the training infrastructure for better efficiency and reliability.
• Build and improve developer tooling that makes the team more productive.
• Work across the full stack, from low-level optimization to high-level model design.

Requirements

• Hold an MS or PhD in Computer Science, Machine Learning or a related field.
• Have strong software engineering skills and a record of building complex systems.
• Be expert in Python and have used deep learning frameworks, with PyTorch preferred.
• Have experience with large-scale machine learning, ideally for language models.
• Balance research goals against practical engineering limits.
• Solve problems well and focus on results.
• Communicate clearly and work well in a collaborative setting.
• Care about the societal impact of your work.
• Preferred: experience with high-performance, large-scale ML systems.
• Preferred: familiarity with GPUs, Kubernetes and OS internals.
• Preferred: experience with language modeling using transformer architectures.
• Preferred: knowledge of reinforcement learning techniques.
• Preferred: background in large-scale ETL processes.
• You are a strong fit if you have significant software engineering experience and are results-oriented, with a bias toward flexibility and impact.
• You are a strong fit if you will take on tasks outside your job description to help the team, enjoy pair programming and collaborative work, and want to learn more about machine learning research.
• You are a strong fit if you want to work at an organization that operates as one cohesive team on large-scale AI research.
• You are a strong fit if you work on aligning state-of-the-art models with human values and preferences, interpreting deep neural networks, or building new models for these areas.
• You are a strong fit if you see research and engineering as two sides of one coin and want to understand the whole research program to make your insights count.
• You are a strong fit if you have ambitious goals for AI safety and general progress over the next few years and want the best long-term outcomes.

Conditions

Annual base salary is $500,000 - $850,000 USD.

Minimum education is a bachelor's degree or an equivalent mix of education, training and experience, in a field relevant to the role as shown through coursework, training or work. Required years of experience match the internal level for the position.

Expect to be in an office at least 25% of the time under the hybrid policy, with some roles requiring more.

Visa sponsorship is available, but not for every role or candidate. If you receive an offer, the company will make every reasonable effort to obtain a visa and keeps an immigration lawyer for this.

The company is a public benefit corporation based in San Francisco.

You are encouraged to apply even if you do not meet every qualification; the company wants a range of perspectives on the team.

Benefits

• Competitive compensation and benefits.
• Optional equity donation matching.
• Generous vacation and parental leave.
• Flexible working hours.
• An office designed for collaboration.