You and the process

With a person
Anna, in-house recruiter
9 years hiring engineers
30 minutes with a real recruiter
They read your CV with you, on a call, and say where the offers are being lost.
Didn’t find what you were looking for? Tell us what to build
Monte Carlo

Applied AI Engineer

  • Remote
  • 3-6 years

Salary

$200,000 - 260,000/ year

AI summary

For members

The whole posting in a few lines. Sign up to read it here and on every role you open.

Sign Up to Read

Description

The company makes a platform that tells enterprises whether their AI agents can be trusted. Its AI teammate, Bayes, detects, triages, troubleshoots and fixes issues in agents and the data behind them, with the aim of agents that improve after each fix.

More than 400 enterprises use the platform, including Amazon, PepsiCo, CNN, Nasdaq and JetBlue.

The role covers the full path from an open problem through research, prototyping and production. You will be given a problem rather than a spec, and you will work with engineering and data science to integrate the results into the platform. Agent observability went from roadmap to revenue faster than expected, so this work is now on the critical path.

The company created the data observability category and is now building in agent trust, a market still forming. It is Series D, with $236M raised from Accel, Redpoint, Notable Capital, ICONIQ Growth and Salesforce Ventures. Customers include HubSpot, Fox, Nasdaq, Toast and Mercado Libre. The company has been remote-first since day one.

Responsibilities

  • Take an open problem from research and prototyping through to production, and drop approaches that do not work before they become commitments.
  • Design and ship agent-powered features such as root-cause analysis, incident triage and monitor generation, and integrate them into the platform with the engineering team.
  • Build the eval setup that lets these features change safely: golden datasets, regression suites, and offline and online scoring, including deciding what counts as good output.
  • Own retrieval and context pipelines built on customer metadata, lineage and query history. Instrument agent behavior in production, covering traces, failure categories, and cost and latency budgets, to improve quality.
  • Work with data science on detection quality and experiment design, and with product on which actions an agent should take, as opposed to ones it is merely capable of.
  • Set the technical standard for building with LLMs, including patterns, guardrails and internal tools for other engineers to reuse.

Requirements

  • You have built agents that run in production with real users, not just integrated a framework. These agents have real autonomy and internal loops: the model picks tools and decides the next step without a person in the loop, and you kept them running once users arrived. A RAG pipeline with a thin wrapper does not count, and neither does a set of MCP tools over an API.
  • You have run evals and monitored agents after launch. Because agents are non-deterministic, standard tests fall short, so you have owned a framework with golden datasets, regression suites, and offline and online scoring rather than a set of one-off scripts. You have watched agents in production, not only in development.
  • You code in Python every day and are solid on the backend. You do not need to be a distributed systems specialist, but you understand models well enough to reason about their behavior, not just call someone else's API.
  • 3+ years in ML, data science or software engineering, at fast-shipping companies such as startups, AI-native teams or high-growth product companies where releases come every few weeks. If your experience is entirely in large, process-heavy organizations with the platform already built, expect a hard adjustment.
  • You work from a problem rather than a spec. Given an ambiguous problem statement, you design the experiment, build the smallest version that tests it, and move what works into production.
  • You use AI tools daily, including Claude or similar, for writing code and doing research. This is backend and model-layer work with no frontend.
  • You prefer shipping to polishing. Most of this work needs a good answer quickly rather than a perfect one later. You can tell which problems deserve depth and will reject a version that demos well but fails in production.
  • Nice to have: applied statistics and hypothesis testing. Building or maintaining MCP servers. Experience in data and cloud tooling such as Snowflake, Databricks, dbt or Airflow.
  • This role is not a fit if your AI work is RAG with a wrapper or MCP tools over an API that do not decide and act on their own, if your LLM experience is limited to prototypes, notebooks and demos that never handled production traffic, or if you need a fully specified problem or are uncomfortable with ambiguity in a category still being defined.

Conditions

Remote position. Compensation is competitive.

Equality is central to the company's culture, and it is an equal-opportunity employer.

Benefits

  • Equity and competitive compensation.
  • Remote-first working environment.

Where you’d work

Fully remote

You can work from

  • United States

About the company

Monte Carlo

3 of their 3 open roles are remote

Your chances

Worth a look before you spend an evening tailoring a CV for it.

  • 21 checks run
  • 1 red flag

Still hiring?

14 checks

1 red flag

How crowded?

7 checks

No red flags

Fits Me

How well does this role fit you?

Answer a few questions or drop your CV, and every role gets a fit score with the reasons, this one first.

  • Your field
  • Level
  • Stack
  • Work model
  • Salary floor
  • Must-haves
Details10 facts · Role, Location, Compensation
Tech stack
  • Python
  • LLMs
  • RAG
  • dbt
  • Databricks
  • Snowflake
  • Airflow
Equity
Equity offered
Specialty
ML
Region
United States
Pay period
Annual
Show 5 more factsShow less

Role

Category
Data & Analytics
Specialty
ML
Experience
3+ years
Tech stack
  • Python
  • LLMs
  • RAG
  • dbt
  • Databricks
  • Snowflake
  • Airflow

Location

Work model
Remote
Region
United States
Remote from
  • United States

Compensation

Salary
$200,000 - 260,000 / year
Pay period
Annual
Equity
Equity offered

Something wrong with this vacancy?

Similar vacancies

Share this vacancy

What's wrong with it?

The employer never sees who reported.

Reason