Senior Staff ML Engineer, AI Safeguards - Moveworks

Description

The team builds safeguards for the agent harness, covering prompt injection, jailbreaks, toxic content, sensitive topics and leaks of sensitive data.

The same stack serves both commercial and government environments, so detection models are chosen and tuned to work in both.

This is a senior staff role on the AI safeguards work.

Responsibilities

• Create and ship controls in the agent harness that catch prompt injection, jailbreaks spanning one or several turns, toxicity, sensitive topics and sensitive data exposure.
• Write evals that score safeguards on false positives and true positives, drawing on customer reports, adversarial testing and production traffic.
• Partner with the Security team by converting their adversarial findings into eval cases and fixes, and by building mitigations that still work when detection fails.
• Pick, integrate and fine-tune detection models for the shared commercial and government stack.
• Trade off quality, latency and cost for controls that sit in the request path.
• Build configurable safeguard policies that let customers set how sensitive topics are handled.
• Watch safeguard behavior in production and fix any regressions.

Requirements

• Typically 8 or more years of experience building production software, including 3 or more years of applied ML, meaning training, evaluating and deploying models in production systems.
• Hands-on experience building LLM safeguards, such as guardrail models, content classifiers, prompt injection or jailbreak detection, or safety evals.
• Solid grounding in evaluating agent trajectories and text classifiers, with care for the quality of training and evaluation data.
• Working knowledge of how LLM safeguards are bypassed, including indirect prompt injection via tools and documents and multi-turn jailbreaks.
• Excellent Python skills.

Conditions

Work persona categories (flexible, remote, or required in office) are assigned based on the nature of the work and the employee's assigned location.

To determine eligibility for a persona, the company may check the distance between your primary residence and the nearest office through a third-party service.