Senior Data Engineer - Europe

Description

The company is an international insurer focused on luxury goods and property markets, plus high-end professional insurance. It has offices in 9 European countries and a highly international workforce.

The Europe Data Team is a multidisciplinary group across several countries. It covers data engineering, data analysis and data management, builds data pipelines and reporting, and sets up governance, working with business teams across Europe.

The team's aim is to provide reliable, consistent and accessible data that supports better decisions and the company's enterprise data strategy.

Responsibilities

• You will lead the design and growth of the European data platform, which runs on Microsoft Azure and Databricks, making technical choices that fit the company's data strategy and keep solutions scalable, robust, efficient and cost-effective.
• You will own engineering delivery for key platform components, set the engineering standards the team follows, and manage the daily work of a small engineering pod. Appraisal and progression stay with the Data Engineering Manager.
• Platform design: you will design and build complex Databricks pipelines and platform architecture, set the direction for compute, governance and Delta Lake standards, keep the platform scalable, fast and reliable, and lead data modelling to make structures scalable, performant and easy to maintain.
• Platform operations: you will own deployments, CI/CD and observability to keep releases reliable and consistent, lead cost management (FinOps) by finding and delivering savings, resolve complex data and platform problems, and act as the senior escalation point for incidents, making sure they are fixed and followed up on.
• Technical leadership: you will define and enforce data engineering best practices covering standards, patterns and governance, plan and oversee the pod's allocation, sequencing and delivery quality, mentor junior and mid-level engineers with guidance and regular feedback, and help with hiring, onboarding and improving engineering practice.
• Collaboration: you will work with senior stakeholders to turn their needs into technical solutions, deliver end-to-end data solutions with Data Analysts and Data Management, integrate the platform with IT, Architecture and Group Data teams, and assess new technologies, including automation and AI-supported engineering.
• The role comes during a major data transformation in Europe, including the move to Databricks and the retirement of legacy platforms. You will help shape the platform's technical direction while keeping day-to-day operations and existing commitments on track.

Requirements

• Databricks experience is essential: Spark and PySpark, Delta Lake, job orchestration, cluster configuration and performance tuning.
• Experience with Unity Catalog, including data governance and access management within Databricks.
• A deep understanding of data modelling, particularly star schemas, and of ETL processes.
• Advanced SQL and Python skills, plus solid knowledge of Azure Data Services, including Azure Data Factory.
• Experience with CI/CD, version control and deployment automation for data pipelines, using tools such as Azure DevOps and Git.
• Experience with platform monitoring, observability and cloud cost management (FinOps).
• Ability to turn business requirements into scalable technical solutions.
• Strong communication skills, with the ability to explain technical decisions to non-technical stakeholders.
• Experience leading the daily work of a small engineering team.
• 5+ years in data engineering or data and analytics roles.
• Experience leading platform migrations, ideally in the insurance industry.
• A bachelor's or master's degree in computer science, engineering, mathematics or a related field.
• Fluent spoken and written English.
• You take ownership of technical outcomes from start to finish and stay comfortable in complex, cross-functional settings where priorities shift.
• You communicate ideas clearly to both technical and non-technical audiences.
• You balance technical excellence with delivery deadlines and are motivated by developing others and raising the team's technical standard.
• You are resilient, adaptable and keen to keep learning and improving.

Conditions

This is a permanent position.

Appraisal and progression are handled by the Data Engineering Manager.