Member of Technical Staff

Description

About the Role

Build reinforcement learning environments and evaluations for a healthtech team developing clinical decision support tools. You will own the work from task design and data creation through environment implementation and delivery to model-training teams, helping make clinical AI more reliable in practice.

Responsibilities

• Build environments for complex clinical reasoning, including multi-step tasks, tool use, state transitions, and measurable success criteria.
• Design and calibrate reward functions, grading rubrics, and LLM judges, and test them for reward hacking or misleading scores.
• Own task design, data creation, environment implementation, evaluation, and delivery to model-training teams.
• Run post-training and agent evaluation experiments, then use failures to improve environments.
• Collaborate with clinicians and technical leadership to validate model outputs and evaluation quality.

Requirements

• A PhD in computer science, machine learning, or a related field, or at least three years of full-time reinforcement learning research or post-training experience.
• At least three years of research or industry experience, or current final-year PhD or postdoctoral status, with a bachelor's degree or higher in a relevant technical field.
• Hands-on experience building reinforcement learning environments for post-training or agent evaluation, ideally in a frontier AI lab or leading AI company.
• Production-level Python, PyTorch, and Transformers experience, plus experience with LLMs, post-training, and evaluation.
• Experience owning reward functions, grading rubrics, or LLM-judge calibration, and communicating failed experiments and uncertainty clearly in writing.
• Clinical AI experience is useful, particularly in clinical reasoning, longitudinal records, or medical workflows. Knowledge of information retrieval design, constraint solving, SAT/SMT, or domain-specific languages is also useful.
• Comfort working across disciplines and with ambiguity, willingness to attend in-person offsites multiple times per year, and experience validating plausible model outputs.

Conditions

Base salary: $250,000 to $400,000 USD annually. Visa sponsorship is not available.

Remote, based in Canada or the United States. The primary location is Canada.