Quality Assurance Engineer (Automation)

Description

The company is transforming the quality and accessibility of healthcare by automating clinicians’ most time-consuming tasks.

Thousands of clinicians across multiple disciplines use Lyrebird every day in environments where accuracy, safety, and time matter deeply. Everything we build is grounded in earning trust, reducing cognitive load, and delivering meaningful impact at scale.

Responsibilities

• You’ll own the quality of the Lyrebird platform, from the first commit to the release gate. This is a hands-on, code-first engineering role, not manual test scripting. You’ll make our Playwright end-to-end and integration suites faster and more reliable, build contract tests that stop backend changes from silently breaking the web app, and design ways to evaluate clinical model output where exact-match assertions don’t work. You’ll also be the quality partner to our SRE function, giving them the health checks, synthetic monitors, load test results and post-deployment signals they need.
• This role suits an engineer who sees testing as a craft and treats a test suite as production code. You’re comfortable moving between TypeScript and Python in the same day, you get real satisfaction from deleting a flaky test and cutting minutes off a pipeline, and you’d rather help other engineers write better tests than be the only person who writes them. You’ll enjoy the hard problem at the centre of this role: how to prove that non-deterministic AI output is safe and accurate enough for clinicians to rely on.
• Optimise our Playwright end-to-end suite, cutting run time and removing flaky tests for good
• Extend our API and partner integration suites, and build contract tests from our OpenAPI schemas so no backend change can break the web app without warning
• Add coverage gates to our Buildkite pipeline, using reporting that already breaks quality down by product area
• Support and collaborate with our engineering teams with CICD development and oversight
• Test the clinical output of our models with DeepEval, openevals and Langfuse traces, designing evaluations that hold up when responses change from run to run
• Run load tests with Locust ahead of major releases and hand clear findings to the SRE function
• Build the signals SRE depends on: health checks, synthetic monitors and post-deployment tests
• Own the release quality gate, giving a clear yes or no with the evidence behind it
• Raise the standard of testing across engineering through shared patterns, fixtures and thoughtful code review
• What you’ll bring
• Strong test automation experience in both TypeScript and Python, writing tests as production-quality code
• Deep hands-on experience with Playwright, plus component testing with Vitest and Testing Library
• Solid pytest skills, including async tests, fixtures, mocking and test data factories such as polyfactory or Faker
• Experience running tests in CI pipelines (ideally Buildkite) and in Docker, with coverage measurement and reporting
• Working knowledge of SQL and Postgres, and confidence testing at the database layer
• Experience with API contract testing against OpenAPI schemas

Requirements

• Experience evaluating large language model output with tools such as DeepEval, openevals or Langfuse
• Load testing experience with Locust
• Familiarity with Svelte, Storybook, or Supabase and its CLI-based database testing
• Experience with observability tools such as Sentry, OpenTelemetry and Honeycomb
• A background in healthcare, clinical software or other regulated, safety-critical environments
• Every test you write protects a clinician who is relying on Lyrebird in the middle of a busy day, and the patient sitting in front of them. When our platform is fast, stable and trustworthy, clinicians spend less time on admin and more time on care. You’ll be the person who makes sure we can promise that with confidence, release after release.
• We’re building a team that reflects the diversity of the people who benefit from our work. We want Lyrebird to be a place where everyone feels safe, supported, and able to thrive. If you’re from an underrepresented background in tech, we strongly encourage you to apply, even if you don’t meet every requirement.