You and the process

With a person
Anna, in-house recruiter
9 years hiring engineers
30 minutes with a real recruiter
They read your CV with you, on a call, and say where the offers are being lost.
Didn’t find what you were looking for? Tell us what to build
Concurrence

Software Engineer, Voice Evaluation [NYC or SF]

  • Office

Salary

$220,000 - 300,000/ year

AI summary

For members

The whole posting in a few lines. Sign up to read it here and on every role you open.

Sign Up to Read

Description

The company is building a platform that healthcare organizations use to create their own clinical AI agents, with the aim of making good care available to every patient.

Beneath the platform is a unified, event-level record of each patient's care, built from a health system's data. Agents on top of it handle intake, scheduling, medication management and clinical decision support.

The agents have supported more than 10 million patient encounters and are expected to grow tenfold this year. The system runs on Databricks and integrates with every major EHR.

The company has about 35 people and is backed by Madrona, General Catalyst and Optum Ventures.

This role builds the system that decides whether a voice agent is safe to put in front of patients. You will own the realistic audio tests, the scoring and the release gate, and you will have authority to block releases that fail it.

The tests must catch problems a clean transcript can hide, such as badly handled interruptions, missed clinical details and failing to pass a call to a human.

Responsibilities

  • Build simulated callers and call replays that run the production voice path on real audio.
  • Test phone trees, hold queues, interruptions, conflicting clinical information and whether escalation to a person happens when it should.
  • Build scoring systems and check automated judges against human assessments.
  • Measure clinical failure rates and their uncertainty, and find out whether simulation results predict what happens on real calls.
  • Set release criteria with model, runtime, product and clinical colleagues, then run the gate and block any change that misses them.
  • Turn production failures into regression tests, confirm fixes work and watch behavior after release.
  • Keep held-out evaluation sets so models are tested on examples they were not trained on.

Requirements

  • You have built and owned evaluation or simulation systems for production agents at scale.
  • You have built agent simulations or replay systems, implemented scoring, and caught a real failure before release.
  • Your evaluation results decided what shipped, and you led that work.
  • You can explain what a score measures, what it misses and how you checked it against real behavior.
  • You write reliable software that holds up at volume, and you trace failures all the way to a verified fix.
  • You use coding agents in your work and check their output yourself.
  • You are low ego and direct, and you can hold the line on release criteria using evidence.
  • You can work on site in New York City or San Francisco.
  • Voice and audio experience: a strong preference.
  • Experience calibrating automated judges against human labels.
  • Load testing and concurrent calls using audio.
  • Speech or audio machine learning.
  • Experience in healthcare or another regulated environment.
  • You take initiative: when something is broken, you fix it rather than filing a ticket and waiting.
  • You hold yourself to a high bar and want teammates who do the same.
  • You question rules that do not make sense and assumptions that have not earned their place.
  • You say the hard thing, challenge ideas openly and commit fully once a decision is made.

Conditions

The role is based in New York City or San Francisco and is in person.

Benefits

  • Comprehensive health, dental and vision insurance.
  • Daily catered lunch and dinner.
  • Mental health support and wellness coaching.
  • A flexible wellness stipend for fitness, therapy or personal growth.
  • An annual learning budget for courses, books or conferences.
  • A budget for attending conferences.
  • An annual team offsite.
  • Academic collaboration opportunities.
  • Unlimited PTO.

Where you’d work

From the office

About the company

Concurrence

  • Industry: HealthTech

Offices in New York City, United States, San Francisco, United States

Your chances

Worth a look before you spend an evening tailoring a CV for it.

  • 21 checks run
  • 1 red flag

Still hiring?

14 checks

No red flags

How crowded?

7 checks

1 red flag

Fits Me

How well does this role fit you?

Answer a few questions or drop your CV, and every role gets a fit score with the reasons, this one first.

  • Your field
  • Level
  • Stack
  • Work model
  • Salary floor
  • Must-haves
Details8 facts · Role, Location, Compensation, Employment, Company
Type
Full-time
Industry
HealthTech
Region
United States
Pay period
Annual
Show 4 more factsShow less

Role

Category
Development

Location

Work model
Office
Region
United States
Offices
  • New York City, United States
  • San Francisco, United States

Compensation

Salary
$220,000 - 300,000 / year
Pay period
Annual

Employment

Type
Full-time

Company

Industry
HealthTech

Something wrong with this vacancy?

Similar vacancies

Share this vacancy

What's wrong with it?

The employer never sees who reported.

Reason