Lead Data Engineer

Description

The team builds reliable, scalable data products and platforms that help the company make better decisions and get more value from its data.

This is a hands-on leadership role that combines technical work with managing people. You will set the engineering approach and quality standards, and help your team solve complex data problems at scale. The company also wants to expand its use of AI and automation in data engineering, and it welcomes your ideas on working more efficiently.

Externally, this role may also be called Data Engineering Lead, Data Platform Lead or Principal Data Engineer.

Responsibilities

• Lead and grow a pod of Data Engineers, building an environment where people learn, work together and deliver high-quality engineering.
• Set the technical direction for data in your area, guiding data modelling, data warehousing, ETL/ELT and scalable data platforms.
• Stay hands-on by designing and building complex data pipelines and solutions with dbt, Python, SQL and modern cloud data tools.
• Raise the bar on engineering quality by improving testing, data quality, observability and automation so data products are more reliable and easier to run.
• Work with technology and business colleagues to turn complex requirements into practical data solutions, and explore how AI and emerging technologies could improve how your team builds and delivers.

Requirements

• Essential: hands-on experience using dbt to design, build and improve data transformations in a complex data environment.
• Essential: engineering experience with Python and SQL, and the ability to solve complex data problems at scale.
• Essential: the ability to design and guide modern data engineering solutions, with knowledge of data modelling, ETL/ELT, data warehousing and scalable data pipelines.
• Essential: the ability to lead and develop Data Engineers, combining hands-on contribution with coaching, technical direction and ownership of delivery.
• Essential: an interest in AI and automation, and the curiosity to explore how emerging technologies could improve engineering quality, productivity and ways of working.
• A plus: experience with AWS or another major cloud platform. The team uses EMR, Athena, Lambda, S3, Glue and Step Functions, but transferable experience on similar problems is welcome.
• A plus: experience with Snowflake, Spark, Scala or Airflow, or with comparable technologies on similar data engineering challenges.
• A plus: experience with Git, GitHub and CI/CD practices, including tools such as GitHub Actions.
• A plus: knowledge of real-time data processing, data governance, data quality or observability.
• A plus: experience using AI-enabled engineering tools or automation to improve data engineering productivity, quality or reliability.
• Candidates who do not meet every requirement are still encouraged to apply if they have strong data engineering foundations, hands-on dbt experience and experience solving complex problems at scale, even without every listed technology.

Conditions

This is a permanent role with hybrid working.

The company is an equal opportunity employer. It will make reasonable adjustments for candidates with disabilities or caring responsibilities during the application and interview process.