Senior Data Engineer

Description

You will join the Core Data Services team, which handles data and analytics work for a live entertainment and eCommerce company. The company runs a global ticketing marketplace, produces over 20,000 shows a year, manages more than 250 artists and runs an entertainment marketing network reaching 200 million consumers annually.

The team is building a modern enterprise Data Lake while keeping current data platforms running. Its aim is trusted, usable data for engineers, data scientists, analysts, product teams and decision makers, which helps the company serve fans and the business.

This senior role centres on large-scale data systems on Databricks and SQL Server, with AI tools and automation used to speed up development and operations.

Responsibilities

• Build and maintain scalable ETL and ELT pipelines with Databricks, PySpark, Python and SQL.
• Tune data workflows and SQL and Spark queries for speed, cost and reliability.
• Help shape the enterprise Data Lake and the curated datasets built on it.
• Create strong ingestion frameworks, transformation logic and reusable engineering components.
• Support migrations that move data and workloads from legacy systems to cloud platforms.
• Use AI coding copilots such as Databricks Assistant, GitHub Copilot and SQL copilots to speed up coding, reviews, tests and documentation.
• Automate routine work such as schema changes, config generation, logging templates and test data with generative AI.
• Use AI to spot pipeline anomalies, data drift, quality problems and performance bottlenecks.
• Apply AI-assisted query tools to improve Spark and SQL performance.
• Use LLM-based tools to generate, refactor and explain complex SQL, PySpark logic and pipeline configurations.
• Work with product teams and data consumers to deliver high-quality data solutions.
• Take part in Agile ceremonies, sprint planning and technical design discussions.
• Mentor junior data engineers and share good practice for AI-assisted data engineering.
• Keep documentation strong and help the team adopt standards and improve continuously.

Requirements

• 5, 7+ years of hands-on data engineering experience.
• Advanced skills in Python, SQL, PySpark and ETL or ELT frameworks.
• Strong experience with Databricks and distributed data processing.
• A record of tuning SQL and Spark workloads for performance and cost.
• Experience with AI development tools such as GitHub Copilot, Databricks Assistant, SQL or IDE AI copilots, and AI-based data quality or monitoring tools.
• Solid grasp of data modelling, governance and CI/CD for data pipelines.
• Clear communication and the ability to work across functions.
• Experience in Agile teams using Jira and Confluence.

Benefits

• Medical, vision and dental cover for you and your family, plus flexible spending accounts (FSA) and health savings accounts (HSA).
• Generous paid time off, including paid holidays, sick time and a paid day off for your birthday.
• Free concert tickets.
• 401(k) with company match and a stock program.
• Parental programs, including caregiver leave, childcare cash and infertility support.
• Tuition reimbursement, student loan repayment, internal growth programs and training.
• Volunteer time off and a crowdfunding network for colleagues.