Staff Software Engineer in Test (Modeling and Simulation)

Description

The company builds automated software engineering tools for teams that need code they can trust. It limits AI to tasks it handles reliably, checks each step independently of the model using formal methods, and keeps engineers involved in key decisions, producing code with auditable proof of correctness. Founded in 2023, it has offices in Boston and San Francisco. Customers include the U.S. Air Force, L3Harris, RTX, and Toshiba. Investors include Accel, Salesforce Ventures, B Capital, Smith Point Capital, J2 Ventures, Shield Capital, Overmatch, RTX, and others.

More information is available at the company's site.

The company is hiring a founding Staff Quality Engineer (SDET) for Modeling & Simulation. The role sits in the central Quality Engineering function, which is working to make software quality measurable, repeatable, and actionable.

The Modeling & Simulation team builds a wargaming and decision-support platform for the U.S. Department of War to assess mission- and campaign-level force-design choices. The system includes backend APIs, asynchronous workers, geospatial and sensing models, authenticated multi-team play with fog of war, external simulation integrations, and AI-assisted features.

The platform already has automated unit and integration tests, but its distributed services, stateful simulation, external integrations, probabilistic behavior, and high-consequence workflows need a more systematic validation design. This role extends that foundation into reproducible quality evidence across services, models, and end-to-end workflows.

This is a hands-on role. You will design and build the automated validation system, write critical tests, and help engineers build quality into their work.

Responsibilities

• Set a risk-based validation approach for Modeling & Simulation covering unit, integration, contract, end-to-end, performance, resilience, and security-critical workflows. Decide which behaviors need deterministic verification, statistical or model-based evaluation, or human review, and how limitations and residual risk are reported as release evidence.
• Build dependable test infrastructure: deterministic fixtures, isolated test data, service and event-contract tests, integration harnesses, versioned reference scenarios, deterministic replay, and browser automation for key user journeys.
• Work with shared platform owners to speed up CI through test partitioning, parallel runs, caching, failure attribution, and flaky-test control.
• Write behavior-focused tests for high-risk planning, sensing, simulation, authorization, state-transition, and recovery logic.
• Work with simulation and model owners to turn intended behavior, assumptions, and known limits into reusable reference scenarios, regression baselines, characterization or differential tests, and reproducible evidence.
• Define validation and release criteria for simulation and AI-assisted workflows, combining deterministic assertions, contracts, reference checks, statistical or model-based evaluation, failure-recovery tests, and human oversight.
• Build repeatable performance and reliability validation with measurable goals and clear release evidence.
• Review designs and code, flag testability risks early, and coach engineers on testing complex simulation, distributed-system, and AI-assisted changes without becoming an approval bottleneck.
• Carry lessons from Modeling & Simulation into shared quality standards, reporting practices, testing guidelines, and validation infrastructure for the wider Quality Engineering function.

Requirements

• Significant experience as an SDET, test-infrastructure engineer, quality architect, or software engineer responsible for automated validation of production systems.
• Solid production experience with Python.
• Deep experience building Python test infrastructure with pytest or a similar framework, including async testing, fixtures and factories, parallel execution, and CI integration.
• Proven ability to design risk-based test strategies across several layers of a distributed system.
• Experience validating systems with partly nondeterministic behavior, such as simulation, statistical models, ML, or AI-assisted workflows, with sound judgment on deterministic checks, reference evidence, statistical evaluation, and human oversight.
• Hands-on testing of Docker-based or similar containerized, multi-service applications, including APIs, databases, messaging, authentication, authorization, and data isolation.
• Experience building and improving CI workflows with GitHub Actions or a comparable platform, including test isolation, failure diagnosis, and flaky-test control.
• Ability to set technical direction, write design documents, mentor engineers, and lead multi-team initiatives through influence.
• Typically 8+ years of relevant software engineering, test infrastructure, or quality engineering experience with Staff-level scope and impact. Depth of experience counts more than an exact year count. You have turned quality strategy into test architecture for a product area, led that work through design docs and influence rather than people management, and shipped the infrastructure and tests yourself.
• Eligible to obtain and keep a U.S. Secret security clearance.
• Preferred: experience validating simulation, geospatial, event-driven, or other stateful systems, including reproducible deterministic and probabilistic behavior.
• Preferred: familiarity with testing or debugging C++ simulation or systems code.
• Preferred: experience testing TypeScript and React applications with Playwright or equivalent tools.
• Preferred: experience with performance testing, generated test data, and API or schema validation.
• Preferred: experience testing cloud-based containerized systems using logs, metrics, traces, and retained test artifacts.
• Preferred: familiarity with constructive simulation, distributed simulation standards, or mission-analysis software.
• Preferred: hands-on evaluation of AI-assisted or agentic systems, including tool-calling and multi-step workflows, with an understanding of risks such as nondeterminism, unsupported outputs, prompt or context manipulation, unsafe tool use, hidden state, and silent failure.
• Preferred: experience using AI coding assistants or agents across the development lifecycle and defining quality controls for AI-assisted changes, such as scoped workflows, independently verifiable tests, human code review, CI checks, and risk-based release evidence.
• Preferred: experience in a regulated, high-assurance, or mission-critical domain.

Conditions

Flexible hybrid or remote work arrangement.

Pay depends on experience, and the company aims for the upper end of the salary range.

The salary range is not a guarantee of pay. The final offer may vary based on factors such as proficiency, skills, experience, and location.

U.S. citizenship may be required for certain project assignments that involve a security clearance.

The company will consider qualified applicants with arrest and conviction records, in line with the San Francisco Fair Chance Ordinance.

The company is an equal opportunity employer.

Benefits

• Health care plan with 100% premium coverage for medical, dental, and vision.
• 401(k) with 5% matching.
• Uncapped vacation, plus sick leave and public holidays.
• Relocation assistance for qualifying employees.