Forward Deployed AI Engineer - Assistant Vice President

Description

The Applied AI and Agents Factory is a group of engineers who turn advanced AI into production systems. You will work alongside business and technology teams, solve hard engineering problems, build intelligent agents, and move ideas from first concept to live use.

The work goes past chat tools. You will build systems that reason, plan, and carry out multi-step workflows, and you will own the whole engineering lifecycle, including agent architecture, orchestration, enterprise integration, security, and observability.

The Factory values hands-on building, teamwork, and quick delivery. You can tackle difficult technical problems, shape architecture decisions, and help create reusable tools and engineering standards used across the company.

The Forward Deployed Engineer (C12) is a hands-on software engineer on an agile team embedded in Line of Business pods. You will build agent features and prototypes, create automated evaluation testbeds, tune prompts and data parsers, and automate continuous testing so production AI agents stay accurate, safe, and reliable.

Responsibilities

• Build and refine agent features, prompt templates, system instructions, few-shot examples, and data pre-processing scripts, working under senior team members.
• Create, calibrate, and run automated evaluation suites in enterprise evaluation studios and frameworks, measuring task completion, tool call correctness, semantic similarity, latency, and hallucination rates.
• Curate, version, and maintain domain-specific golden datasets, synthetic benchmarks, and edge-case regression testbeds with Line of Business domain experts.
• Write and maintain automated test scripts in enterprise CI/CD pipelines that enforce continuous evaluation gates before staging and production releases.
• Test AI guardrails, including prompt injection defenses, toxicity filters, and PII redaction, across agent inputs and outputs to confirm safe behavior.
• Track agent performance, log evaluations, and write prompt diagnostic reports that help squad leads and business stakeholders improve agent behavior.
• Take on more independence over time within fast-paced agile squads, contributing to sprint planning, standups, and codebase upkeep.
• Assess risk in business decisions with attention to the firm's reputation and to protecting the company, its clients, and its assets. Drive compliance with applicable laws, rules, and regulations, follow Policy, use sound ethical judgment in your conduct and business practices, and escalate, manage, and report control issues openly.

Requirements

• 5+ years of relevant software engineering experience, or 2+ years with a specific focus on Python, Java, or Go and Generative AI application development.
• Strong Python, Java, or Go skills, with hands-on work integrating REST or gRPC APIs, Docker containers, and Git workflows.
• Working knowledge of Generative AI concepts, prompt engineering, and evaluation methods such as semantic similarity, LLM-as-a-judge, and benchmark test suites.
• Experience with automated test pipelines and CI/CD tools such as Tekton, Jenkins, GitHub Actions, or Harness.
• Strong analytical, problem-solving, and communication skills, with a methodical, detail-oriented approach to software quality and data integrity.
• Bachelor's degree or equivalent experience in Computer Science, Data Science, or a related quantitative field.

Conditions

Full-time role based in New York, New York, United States.

Base salary range is $121,200.00 to $181,800.00.

Incentive and retention awards may be offered to eligible employees, in addition to salary.

AI may help match your skills to openings, but people conduct, check, and decide all hiring outcomes.

Benefits

• Medical, dental, and vision coverage.
• 401(k) plan.
• Life, accident, and disability insurance.
• Wellness programs.
• Paid vacation, paid sick leave, and paid holidays.
• Benefits and availability may vary by jurisdiction, job level, and hire date.