You and the process

With a person
Anna, in-house recruiter
9 years hiring engineers
30 minutes with a real recruiter
They read your CV with you, on a call, and say where the offers are being lost.
Didn’t find what you were looking for? Tell us what to build
Aleph Alpha

AI Researcher - Pre-training (f/m/d)

  • Office

Salary

Not stated

Similar roles pay €65K - 115K a year · our estimate

AI summary

For members

The whole posting in a few lines. Sign up to read it here and on every role you open.

Sign Up to Read

Description

The company is one of the few in Europe that pre-trains foundation models from scratch, and it recently released a model of its own.

Its customers in finance, manufacturing and public administration need models that handle German, follow European regulations and perform reliably where errors are costly.

The team is based in Heidelberg and works on the architecture and training of the next generation of foundation models.

You get hands-on access to large clusters with thousands of NVIDIA Blackwell GPUs, and your experiments feed into production training runs.

The team is flat, with little hierarchy, and people often pair or mob on hard problems. New hires start on scoped problems with senior researchers and take on more ownership over time.

Responsibilities

  • Design and run ablations on parts of the training recipe such as optimizers, learning rate schedules and initialization.
  • Implement architecture changes in PyTorch and measure their effect on convergence, stability and training efficiency.
  • Run small proxy experiments and help build the hyperparameter scaling laws and scale-up methods used to predict behaviour at multi-thousand-GPU scale.
  • Help monitor large training runs and investigate loss spikes, divergence and distributed training failures.
  • Read the literature, reproduce promising results at small scale and bring the most useful ideas to the team.
  • Work with the Compute Performance, Data, Evaluation and Post-Training teams on how hardware limits and the rest of the model lifecycle affect pre-training choices.

Requirements

  • Proficiency in Python and hands-on experience with PyTorch.
  • A strong machine learning foundation, shown through a PhD, a strong master's thesis, publications, open-source work or substantial professional or personal projects.
  • A solid background in linear algebra, probability and optimization, and comfort reasoning formally about why models train as they do.
  • A working understanding of transformers and experience pre-training an LLM from scratch, even at small scale.
  • Careful experiment design, and honest reporting of noisy and negative results, including what your evidence does and does not show.
  • A record of learning unfamiliar technical areas fast and going deep, for example by reimplementing papers, getting up to speed on a large codebase or debugging far outside your usual area.
  • Clean, maintainable code and care for reproducible experiments.
  • Clear communication, raising problems early and actively seeking and acting on feedback.
  • Preferred: experience training on multi-GPU or multi-node setups, or with distributed frameworks such as Megatron-LM, DeepSpeed or torchtitan and data, tensor, pipeline or expert parallelism.
  • Preferred: familiarity with scaling laws, hyperparameter transfer such as µP, or recent work on optimizers and training stability.
  • Preferred: exposure to Mixture-of-Experts, efficient attention variants or other inference-efficient designs.
  • Preferred: publications at top venues such as NeurIPS, ICML or ICLR, or contributions to open-source ML training libraries.
  • Preferred: a strong curiosity about the hardware and systems limits that shape scale. Low-level kernel optimization is not required.

Conditions

The role is based in Heidelberg.

Where you’d work

From the office

About the company

Aleph Alpha

  • Industry: AI

Office in Berlin, Germany

Also hiring in Germany

Your chances

Worth a look before you spend an evening tailoring a CV for it.

  • 19 checks run
  • 1 red flag

Still hiring?

13 checks

1 red flag

How crowded?

6 checks

No red flags

Fits Me

How well does this role fit you?

Answer a few questions or drop your CV, and every role gets a fit score with the reasons, this one first.

  • Your field
  • Level
  • Stack
  • Work model
  • Salary floor
  • Must-haves
Details9 facts · Role, Location, Compensation, Employment, Company
Type
Full-time
Industry
AI
Specialty
ML
Region
Europe
Pay period
Annual
Show 4 more factsShow less

Role

Category
Data & Analytics
Specialty
ML

Location

Work model
Office
Region
Europe
Office
  • Berlin, Germany

Compensation

Salary
Salary by agreement
Pay period
Annual

Employment

Type
Full-time

Company

Industry
AI

Something wrong with this vacancy?

Similar vacancies

Share this vacancy

What's wrong with it?

The employer never sees who reported.

Reason