Lead Product Manager, Agent Runtime & Evals

Description

The company is an operations partner for private markets, serving more than 2,300 GP firms with a single platform that combines technology, data, and fund administration. It has over $300B under administration, more than 700,000 LPs on the platform, has been founder-led since 2014, has raised more than $350M, and employs over 1,000 people. Its AI platform, JunieAI, embeds intelligence across workflows.

The company works across 27 U.S. states, 2 Canadian provinces, India, Luxembourg, and England, with offices in San Francisco, New York City, Mumbai, and Bangalore. People can work fully remote or in an office, all or part of the time.

The role owns the agent runtime and evaluation layer of Foundry, where non-engineers build, deploy, and run AI agents, apps, and code. Each automation gets its own identity, access grants, and audit trail. You report to the Group Product Manager for Foundry and work with a dedicated engineering team.

About 75% of the work is the runtime: versions, deployments, evals, testing, and how new models are adopted selectively across automations. About 25% is working with the workflow orchestration PM on the contracts that let hosted agents run inside governed workflows. Orchestration sets the container; this role decides how it gets filled with predictable automations.

The platform is early. Fund administration is being rebuilt around agents and humans building and deploying automations together. You will help set foundations and make the call on unclear platform tradeoffs.

You are expected to use modern AI tools daily, such as LLMs, agent frameworks, and Claude, to move faster and decide better. That can include building agents on your own platform, prototyping before writing specs, or using coding agents to open pull requests.

In-person work in San Francisco is preferred, where much of the product, design, and engineering team sits. Strong candidates elsewhere in the United States and Canada are also welcome.

Responsibilities

• Define the vision, strategy, and roadmap for a runtime that lets business users move an agent, app, or script from idea to governed production without an engineering cycle, covering draft, test, publish, version, deploy, run, observe, improve, and retire.
• Set how every automation is tested before release. Work with engineering and the shared eval platform team on golden datasets, labeling, and regression gates that block promotion. Decide when an automated judge is enough and when a person must stay involved. Shape how evals decide which models suit which agents, so a new model release is a product choice.
• Make sure agents that grade other agents are themselves checked. Work with engineering on ways to keep quality measurable across thousands of agents, and present these as clear, trustworthy views for builders.
• Define how agents keep long-term memory, compress it, and learn from experience, judged by adoption and user impact more than by algorithm quality.
• Own model routing, cost attribution, and pricing so teams can pick the best model for each task and see its cost.
• Define how hosted agents reach data and take action across the product suite, fund administration systems of record, and third-party tools. Own the connector, credential, and access-grant model so integrations get reused. Define agent identity and run modes, including when an agent uses the invoking user's access and when it uses its own service identity.
• Design oversight tools for fund administration teams running large fleets of agents: dashboards, run history, traces, metrics, logs, kill switches, and audit trails. Make clear which events are exceptions and which are issues.
• With the workflow orchestration PM, define the agent registry and typed contracts describing what each agent needs and produces, and how workflows bind to them. Keep the safety rule that agent steps stage changes and people release them.
• With design and engineering, build tools that let SMEs configure, test, and tune automations on their own. Keep configuration minimal so every setting justifies itself.
• Track managed agent runtimes, agent frameworks, and automation platforms, and turn that into clear build, buy, or partner decisions.
• Set direction across the platform and adjacent teams, raise the bar for product quality, and be the person engineering relies on for unclear platform tradeoffs.

Requirements

• 7+ years of product management in B2B or enterprise SaaS, including 3+ years in an AI-oriented role.
• A track record owning a complex product area end to end, from strategy and roadmap through delivery and iteration, ideally starting from zero.
• Working knowledge of agent evaluation: golden datasets, labeling, automated judges, regression testing, and when a human must intervene.
• Experience building agents or replacing human-run processes with product and automation, with real evaluation and monitoring in production.
• A strong engineering partner with a product mindset. You can go deep on typed contracts between agents and workflows, structured and deterministic outputs, and agent memory, and you know which questions to ask even without the answers. This is not a TPM role; your focus is how people actually use these tools.
• Empathy for non-engineer builders and operators, including how they oversee large numbers of automated agents.
• Fluency with LLMs and common AI tools such as Claude and Cursor. You can run your own prototypes and get work done faster than a traditional PM process allows.
• Strong written and verbal communication, able to explain product decisions to engineers, designers, executives, and customers.
• A record of leading through influence, ideally mentoring or managing product managers, and setting strategy that several teams carry out.
• Nice to have: deep experience with managed agent platforms such as Claude Managed Agents or Workato Genies, including evals, tool and connector access, versions, and deployments.

Conditions

The U.S. base salary range is $200,000 to $230,000 USD, plus equity and benefits. The actual base depends on experience, skill set, location, and any local minimum pay rules.

The company provides equal opportunity in employment.

Benefits

• Health, dental, and vision coverage for you and your family.
• Life insurance and mental wellness coverage.
• Fertility and family-building support.
• Flex time off plus company-paid holidays.
• Paid family, medical, and bereavement leave.
• Retirement savings plans.
• A home work and technology setup allowance.
• An annual professional development stipend.
• Your recruiter can share more detail on pay and benefits.