You and the process

With a person
Anna, in-house recruiter
9 years hiring engineers
30 minutes with a real recruiter
They read your CV with you, on a call, and say where the offers are being lost.
Didn’t find what you were looking for? Tell us what to build
Bristol Myers Squibb

Senior Manager, AI Engineer/ Solutions & Engineering

  • Hybrid
  • 6+ years

Salary

$153,770 - 186,327/ year

AI summary

For members

The whole posting in a few lines. Sign up to read it here and on every role you open.

Sign Up to Read

Description

The company discovers, develops and delivers medicines for serious diseases, and digital and AI work is central to its plans.

AI is being used to speed up content creation and decisions, and to personalise customer engagement, increasingly through enterprise AI assistants and agents that commercial teams use every day.

This is a hands-on role for an AI engineer who solves real business problems with large language models (LLMs).

You will shape how models are prompted, grounded, given tools, evaluated and governed, and you will build the solutions that run on them, using coding agents to work quickly.

You need enough understanding of models and agent harnesses to predict where a solution will succeed or fail, and to design around both.

Requisition R1608179, Senior Manager, AI Engineer/ Solutions & Engineering.

Responsibilities

  • Build and launch LLM solutions, such as assistant extensions, connectors, agent workflows and custom apps, from prototype to production.
  • Write evaluations before you build: define success for each use case, create golden datasets, graders and regression suites, and use them to guide every change to prompts, tools and models.
  • Use coding agents such as Claude Code every day: write specs, direct the agents, and review and test their output, remaining responsible for its correctness and quality.
  • Account for the harness by learning how AI assistant platforms run the agent loop, including tool calls, context assembly, permissions and sandboxing, and shape solutions to fit.
  • Manage context deliberately through system prompts, reusable instructions, memory, retrieval, structured outputs and long-context strategy.
  • Pick the build approach (platform extension, connector, custom app or API-based agent) and decide when an agent is the wrong choice.
  • Choose models by weighing capability, cost, latency and risk, backed by your own evals.
  • Add guardrails and human review suited to regulated pharma content, covering MLR review, compliance, auditability and data boundaries.
  • Track quality, cost, latency and adoption of live solutions using production traces and feedback, and improve them.
  • Work with marketing, commercial and IT partners to turn business needs into technical solutions.
  • Help the business understand what AI can do, where it fails and what it is worth.
  • Raise the team's practice by sharing reusable patterns, reviewing peers' work and keeping up with new models and platform features.

Requirements

  • Bachelor's degree in computer science, engineering, data science or a related field, or equivalent experience.
  • 5+ years of software or ML engineering experience, including at least 1 year building LLM or agent-based solutions.
  • A portfolio of shipped work, such as assistant extensions, connectors, agent workflows or an eval suite you built.
  • Strong communication skills, including explaining trade-offs to business partners.
  • Evaluation: design evals for non-deterministic systems using golden datasets, rubric and LLM-as-judge graders, human review and regression tests.
  • Evaluation: assess retrieval, tool use, multi-step agent trajectories and final output as separate layers.
  • Evaluation: calibrate LLM graders against human labels, spot grader bias, run evals in CI so changes are measured before release, and turn results into fixes through error analysis.
  • Coding agents: hands-on daily use of Claude Code, Codex, Cursor or similar to design, build, test and refactor software.
  • Coding agents: write clear specs and acceptance tests, split work into agent-sized tasks, and run agents in parallel or on long tasks.
  • Coding agents: configure project instructions, skills, MCP servers, hooks and permissions, and review output critically so you can defend every line you ship.
  • Applied LLM engineering: proven LLM solutions in production, not only prototypes.
  • Applied LLM engineering: strong Python, with TypeScript ideally as well, API-driven development, Git, CI/CD, and a cloud platform (AWS, Azure or GCP).
  • Applied LLM engineering: advanced prompt and context design, including few-shot patterns, structured outputs, reasoning and long-context strategies.
  • Applied LLM engineering: retrieval and grounding with chunking, hybrid search, reranking and metadata filtering, and knowing when connectors or direct file access can replace a vector database.
  • Applied LLM engineering: agentic patterns such as tool calling, multi-step workflows and MCP, with judgment on when not to use an agent.
  • Applied LLM engineering: working with unstructured and semi-structured data such as documents, PDFs and content assets.
  • Models and harnesses: transformer basics, including tokenization, attention, context windows and sampling.
  • Models and harnesses: the training stack (pretraining, instruction tuning, RLHF/RLAIF) and how reasoning models differ.
  • Models and harnesses: how an agent harness works, including the loop, tool exposure, context management and permissions.
  • Models and harnesses: failure modes such as hallucination, sycophancy, position bias, prompt injection and tool misuse, and how to mitigate each.
  • Models and harnesses: reasoning about cost, latency and quality from first principles.
  • Engineering practice: LLMOps, including tracing, observability, versioning of prompts and models, and cost and usage management.
  • Engineering practice: security basics, including prompt injection through tools and documents, data leakage, PII and least-privilege access.
  • Desirable: hands-on experience building on enterprise AI assistant platforms such as Claude Desktop (Chat, Cowork, Code, with skills, projects, plugins and connectors), ChatGPT Enterprise (custom GPTs, connectors, projects), Gemini Enterprise or Gemini for Workspace, or similar harnesses.
  • Desirable: building or configuring MCP servers and integrating enterprise systems as connectors.
  • Desirable: agent SDK or API-based agent development for cases that outgrow the assistant platform.
  • Desirable: experience in pharma, life sciences or other regulated industries, including MLR, GxP and 21 CFR Part 11 awareness.
  • Desirable: knowledge of commercial pharma data and systems (Veeva, PromoMats, IQVIA), digital marketing, content supply chain, or HCP and DTC engagement.
  • Desirable: enterprise AI rollout work, such as adoption programs, workshops, champion networks and usage governance.
  • Desirable: multi-agent or sub-agent orchestration, for example with LangGraph.
  • Desirable: multimodal and document-understanding workloads.
  • Desirable: fine-tuning or distillation (LoRA, PEFT) and open-weight models (Hugging Face, Llama, Mistral).

Conditions

This is an on-site role based in Princeton, NJ, US.

Full-time starting pay ranges from $153,770 to $186,327.

Full-time employees may also be eligible for discretionary incentive cash and stock.

Starting pay depends on the role, required skills, work location, schedule and your experience, and is guided by pay equity principles and applicable law.

Compensation programs are reviewed over time and may change with market factors and individual, team or company performance.

The company uses four work models (site-essential, site-by-design, field-based and remote-by-design), and this role's model follows its core responsibilities.

Candidates who do not match every item are encouraged to apply anyway.

Applicants with arrest or conviction records are considered under applicable law, and reasonable accommodations are available during recruitment.

Benefits

  • Eligible employees can join comprehensive benefit plans, subject to plan terms, including wellbeing support, retirement and financial protection, and insurance for medical, dental, vision, life and disability.
  • US-based exempt employees get Flexible Time Off with no set accrual limit, subject to manager approval, plus 11 paid holidays each year.
  • Non-exempt employees, RayzeBio employees and employees in Puerto Rico receive 160 hours of paid vacation per year for new hires, subject to manager approval, 11 paid holidays and 3 optional holidays.
  • Depending on eligibility, you may get paid sick leave, up to two paid volunteer days per year, summer hours flexibility, and leave for medical, personal, parental, caregiver, bereavement or military needs.
  • Eligible employees also get a company-wide shutdown from Christmas Day to New Year's Day.
  • Full US benefit details are published on the company's careers site.

Where you’d work

Part of the week in the office

You can work from

  • United States

About the company

Bristol Myers Squibb

  • Industry: HealthTech

Office in United States

Also hiring in Ireland, Netherlands, Munich, Germany and 11 more places

22 of their 42 open roles are remote

Your chances

Worth a look before you spend an evening tailoring a CV for it.

  • 22 checks run
  • 1 red flag

Still hiring?

15 checks

1 red flag

How crowded?

7 checks

No red flags

Fits Me

How well does this role fit you?

Answer a few questions or drop your CV, and every role gets a fit score with the reasons, this one first.

  • Your field
  • Level
  • Stack
  • Work model
  • Salary floor
  • Must-haves
Details13 facts · Role, Location, Compensation, Employment, Company
Tech stack
  • TypeScript
  • Python
  • AWS
  • Azure
  • CI/CD
  • LLMs
  • LangChain
  • Vector databases
Seniority
Senior
Type
Full-time
Industry
HealthTech
Specialty
ML
Region
United States
Show 7 more factsShow less

Role

Category
Data & Analytics
Specialty
ML
Seniority
Senior
Experience
5+ years
Tech stack
  • TypeScript
  • Python
  • AWS
  • Azure
  • CI/CD
  • LLMs
  • LangChain
  • Vector databases

Location

Work model
Hybrid
Region
United States
Office
  • United States
Remote from
  • United States

Compensation

Salary
$153,770 - 186,327 / year
Pay period
Annual

Employment

Type
Full-time

Company

Industry
HealthTech

Something wrong with this vacancy?

Similar vacancies

Share this vacancy

What's wrong with it?

The employer never sees who reported.

Reason