Senior Applied Machine Learning Scientist, Generative AI Model Validation ATH# 5480

Description

The AI Model Validation group sits within the bank's centralized model risk management function. It has grown quickly in recent years as global regulators have paid more attention to model risk.

The group reviews, vets and approves AI models built and used across the enterprise, including business lines such as fraud, retail credit risk, marketing, NLP, wealth, asset management, treasury, insurance and securities. The focus covers generative AI, agentic AI and other deep learning models.

This role reports to the AVP of AI Model Validation and centers on leading a team that validates generative AI, NLP and other deep learning models.

Responsibilities

• Lead research and development in evaluating and testing generative AI, agentic AI and LLM systems.
• Lead a team of machine learning scientists who validate complex AI models, especially generative AI, LLM and NLP and deep learning models.
• Review, test and evaluate AI models, and challenge model developers effectively.
• Recommend whether models should be approved, or what corrective action is needed, based on independent validation.
• Lead and support a team of model validators.
• Set performance objectives for all staff and give them regular performance feedback.
• Share group objectives and strategy, and align team activities with business goals.
• Support professional development, coaching and guiding direct reports toward their development goals.
• Take a leading role in writing standards and procedures for validating generative AI, agentic AI and varied deep learning models, so they follow the bank's model risk policy, industry and academic best practice, and regulatory requirements.
• Answer requests and provide information from Canadian and U.S. regulators, internal audit and external audit as they review models and validation processes.
• Work with internal model development groups, audit and other internal partners to make sure models meet the bank's standards for use.
• Help ensure AI models are used appropriately. Spot when new models or techniques are needed as industry standards and regulations change.
• Keep up deep knowledge of AI and machine learning methods, share it with business partners and senior management, and act as subject matter expert for business units on generative AI, agentic AI, LLM, NLP and deep learning modeling and validation.
• Track developments in generative AI, including key publications, major LLMs, evaluation metrics, technology stacks and datasets.
• Explain findings and recommendations to both technical and non-technical audiences.
• Work effectively with model development groups and other second line risk control functions.

Requirements

• Advanced quantitative and AI skills, backed by a post-secondary degree in a field such as computer science, machine learning or AI, engineering, statistics or mathematics.
• At least 4 years of experience developing or validating generative AI, LLM, NLP and other deep learning models, plus at least 2 years leading a small team of machine learning scientists.
• Deep understanding of AI methods and theory, including generative AI, agentic AI, deep learning, modern NLP, retrieval-augmented generation and transformers.
• Hands-on experience with deep learning and generative or agentic AI tools such as PyTorch, LangChain, LangGraph and HuggingFace.
• Strong drive to keep learning about advances in generative AI, agentic AI, machine learning and cloud technology.
• Proficiency in at least one scripting or programming language, such as Python.
• Familiarity with cloud platforms such as Azure or AWS, and big data tools such as PySpark or Hadoop.
• Familiarity with data structures, algorithm design and object-oriented programming principles.
• Knowledge of explainability and interpretability methods for machine learning.
• Strong verbal and written communication skills and stakeholder management ability, including writing and reviewing technical reports.
• Strong critical and analytical thinking.
• Solid time and project management, and the ability to juggle several tasks with minimal supervision.
• Ability to work independently and with others in a dynamic, changing setting.
• Preferred, a plus: publications in relevant conferences or journals, or research in the generative AI or LLM field.
• Preferred, a plus: ability to implement AI algorithms from academic research papers.

Conditions

Location: Montréal, Quebec, Canada.

Hours: 5.

Line of business: Analytics, Insights, and Artificial Intelligence.

Pay range: $156,500 - $90,000 CAD. This reflects a temporary market premium for this role, reviewed each year.

Actual base pay may vary with skills, experience, job knowledge, location and business needs. The company encourages candidates to ask about compensation and discuss it openly with their recruiter.

This role is subject to provincial employment regulation, and each province or territory in Canada may have its own rules.

Equal opportunity: the company values diverse perspectives and fair opportunities for all.

Benefits

• Total rewards include base salary, variable pay, health and well-being benefits, savings and retirement plans, paid time off, banking benefits and discounts, career development, and recognition programs.
• Regular career, development and performance conversations with your manager.
• Access to an online learning platform and a range of mentoring programs.
• Training and onboarding sessions to help you settle into the role.