Lead Technology Product Manager, AI Quality & Evaluation

Description

You will help shape the product features that make AI-powered healthcare tools useful, trustworthy and deployed with care.

The UpToDate suite is used for millions of clinical decisions each day, so the company treats AI quality as ongoing work rather than a single check before launch. AI features must be assessed, watched and refined over time so clinicians and care teams can rely on what they see.

As a Senior Product Manager for AI Quality and Evaluation, you will help turn clinical quality standards and evaluation strategy into product capabilities that scale. You will work under product leadership with clinical, technical and data specialists to define the requirements, workflows, tools, automation and reporting used to assess AI features before and after release.

The clinical quality standards and statistical methods are set by others. Your job is to make them practical through usable tools, repeatable processes, clear measurements and evidence that guides product choices, customer confidence and responsible AI use.

Responsibilities

• Turn quality standards and evaluation methods defined by experts into product requirements, workflows and roadmap priorities.
• Lead discovery and delivery of defined AI quality and evaluation capabilities, such as tooling, automation, monitoring, evidence capture and reporting.
• Work with clinical, data science, engineering, product, legal, compliance and customer-facing teams to put AI evaluation into practice before and after launch.
• Find ways to automate and scale evaluation work while keeping the right level of expert review.
• Write product requirements for review workflows, issue triage, launch readiness, post-launch monitoring and quality reporting.
• Apply evaluation findings to improve the product, user experience, content quality, model behavior and operations.
• Follow emerging market practices and customer expectations on AI quality, evaluation, benchmarking and responsible AI.
• Explain AI quality and evaluation practices clearly to product leaders, customer-facing staff and other internal stakeholders.

Requirements

• At least 5 years of product management experience in healthcare, enterprise SaaS, data-driven products, platform products, quality or related fields.
• Strong skills in product discovery and execution, including writing requirements, planning roadmaps, running experiments and making decisions based on metrics.
• Ability to turn complex clinical, technical, quality or risk concepts into practical requirements and workflows.
• Experience working with engineering, data, clinical, compliance, customer-facing or operations stakeholders.
• Strong analytical skills and comfort defining metrics, dashboards, workflows and decision criteria.
• Clear communication, particularly about quality, trust, tradeoffs, evidence and customer value.
• Curiosity about AI, evaluation methods, responsible AI and new technologies, and the ability to learn quickly.
• Preferred: experience with AI-enabled products, healthcare technology, quality tooling, data products, search or retrieval quality, content quality, clinical validation or decision-support products.
• Preferred: familiarity with AI evaluation methods, benchmarks, monitoring, human-in-the-loop review, automation or applied machine learning concepts.
• Preferred: experience in healthcare, clinical workflows, regulated environments or professional-information products.
• Preferred: experience supporting customer-facing narratives about product quality, trust or responsible technology.

Conditions

Pay range: $107,500.00 to $188,400.00 USD, based on the primary location of the role.

This role is eligible for a bonus.

The actual base salary offer depends on factors including skills, experience and the hiring location. A recruiter can share details for the specific location during the process.

Hybrid work.

Applicants may be asked to attend an onsite interview at a the company office.

Interviews must be completed without AI tools or outside help, and virtual backgrounds must be removed. Using AI-generated answers or third-party support during interviews leads to disqualification.

Equal opportunity: candidates are considered without regard to protected characteristics.

Benefits

• Medical, dental and vision plans, 401(k), FSA or HSA, commuter benefits, tuition assistance, vacation and sick time, and paid parental leave.
• Full benefit details are available on request.