Manager, Test Data Engineering Team

Description

The company runs ticketing, concert promotion and sponsorship marketing for the live entertainment industry, and this manager role sits in its core data services team.

You will lead a globally distributed team across the US, UK and offshore, mostly made up of engineers with QA backgrounds, and grow them into an engineering-led function.

The team builds automated data testing and validation within a Databricks platform, and works with teams across North America, Europe and Asia.

Success means data quality checks in every critical pipeline, faster detection and fixing of data issues, no manual QA validation, and frameworks adopted by engineering teams worldwide.

Responsibilities

• Design and build reusable data quality frameworks that plug into Databricks pipelines.
• Create standard validation libraries covering schema, freshness, volume and anomaly checks.
• Set and enforce data quality standards and a shared Definition of Done for pipelines.
• Build scalable synthetic data generation and masking solutions.
• Develop replay frameworks that test pipeline changes against production scenarios.
• Build automated regression harnesses for batch and streaming pipelines, using TDD and iterative, research-led validation.
• Set up monitoring, alerting and SLA/SLO frameworks for data pipelines, and build tools that track data quality and reliability.
• Detect data issues in production before they spread.
• Lead and grow data quality and test data engineers.
• Work with Data Engineering, Product and Analytics teams to build quality practices into their work, and drive adoption of frameworks across global pods.
• Define and carry out a long-term data quality roadmap, and manage multiple roadmaps and shared-service backlogs.
• Set priorities across stakeholders and time zones, and remove manual validation through automation.

Requirements

• 5+ years in data engineering, test data engineering or a related field.
• 2+ years in a technical leadership or management role.
• Strong hands-on experience with Databricks, Spark, distributed data systems, Python and SQL.
• Experience building automated validation or data quality frameworks.
• Hands-on use of data observability or validation tools such as Great Expectations, Deequ or Soda.
• Familiarity with streaming architectures and real-time validation.
• Solid understanding of anomaly detection and large-scale validation patterns, such as hash or Merkle-based validation, or of major TDM tools for profiling and sampling.
• Comfortable juggling several projects at once and setting priorities.
• Experience writing JIRA stories with agile acceptance criteria.
• Excellent analytical skills.
• Track record of launching test data solutions or BI validation applications.
• Versatile, hands-on approach with a focus on execution.
• Optional: AI orchestration experience is a huge plus, especially for automating synthetic data generation, data profiling and pipeline regression testing.
• Optional: experience with user behavior data is a leg up.
• Optional: passion for live music and events is a plus.

Conditions

Globally distributed team across the US, UK and offshore, requiring cross-time-zone collaboration.

Willingness to work across time zones and odd hours as needed.

The role is highly visible and operates across North America, Europe and Asia.

Benefits

• Health: medical, vision, dental and mental health cover for you and your family, a health care concierge, and FSA or HSA options.
• Personal: free concert tickets, generous paid time off including paid holidays, sick time and personal days.
• Financial: 401(k) with company match and a stock reimbursement program.
• Family: new parent programs including caregiver leave and baby bonuses, plus fertility, adoption, foster or surrogacy support.
• Career: development programs through School of Live, tuition reimbursement and student loan repayment.
• Community: volunteer time off and a crowdfunding match.