Senior Data Engineer Databricks

Description

The team leads data transformation across the company's EMEA and APAC business and is now hiring a Senior Data Engineer.

The Databricks-based Lakehouse is well established and already supports the business end to end.

The next phase is harder: applying agentic capabilities at real enterprise scale across many domains, and helping the company compete in the AI era.

This is a hands-on senior position on major transformation projects in EMEA and APAC. You will work directly with the team lead and a group of analytical engineers who challenge each other to keep up with what is possible.

You will tackle problems with no existing playbook, make architecture choices with lasting impact, and build depth in several domains early enough to shape where the platform goes. Few organisations in the sector have started this work, so the skills you gain here would be hard to get elsewhere in the industry.

Responsibilities

• Lead the move from hand-written code to AI orchestration by building agentic workflows for data engineering tasks and CI/CD, designing semantic models for downstream use, and setting up an AI Gateway for governance.
• Look after the AI-amplified Databricks framework, keeping it running and refining it step by step as you learn.
• Design data models that bring sources from anywhere in the world into one consistent structure.
• Build data solutions for areas such as custom MDM, geocoding, client mastering and profiling. Real estate context is a plus but not required.
• Design and deliver stable, automated ingestion pipelines from varied sources.
• Take ownership of production issues: diagnose and fix pipeline failures, performance problems and data quality faults by methodically working through code, logs and documentation.
• Report on data quality and ship the changes that improve the numbers.

Requirements

• Systems thinking: you design for scale rather than only the ticket in front of you.
• Production troubleshooting and root cause analysis under pressure, working methodically through code, logs and documentation rather than guessing.
• Data modelling: design cross-functional data products, establish data contracts and handle complex business rules.
• Advanced SQL, including window functions, query optimisation and MERGE/UPSERT operations.
• Python and PySpark: write reusable, parameterised functions and work comfortably with JSON, CSV and Parquet.
• Deep experience with Databricks, including Declarative Pipelines, DABS, Delta Lake, Spark optimisation and job orchestration.
• Familiarity with the Azure ecosystem.
• Working understanding of Unity Catalog.
• Around 6 or more years of relevant experience.
• A degree in a quantitative field such as computer science, data science, econometrics, mathematics or physics.
• Drive and curiosity, with a appetite for problems that have not been solved yet.
• Strong written communication, since async updates and clear documentation are core to the role.
• Hands-on experience with agentic data engineering: using coding agents in real delivery and building the harness around them, including repo conventions, reusable context, tests and guardrails that keep agent output trustworthy.
• Geospatial data processing: H3 indexes, spatial SQL and point-in-polygon at scale.
• Recursive CTEs and complex SQL patterns.
• Infrastructure familiarity: Azure Portal, resource management and the CLI.
• Strong Git workflows and code review habits.
• Real estate domain experience.

Conditions

The role covers the company's EMEA and APAC business.

Equal opportunity employer: candidates are considered regardless of ethnicity, gender, age, disability, religion or any other legally protected characteristic.

Benefits

• You set the high-level roadmap, and you decide how to reach it.
• Protected learning time, plus a team that pushes each other forward as the field changes weekly.
• A rare growth path, building capability in an area few organisations in the sector have started, and a skill set hard to assemble elsewhere in the industry.
• An existing technical foundation, with architecture, standards and best practices already in place, so you build on top rather than from scratch.