The team is hiring a senior data engineer to build scalable data pipelines and Lakehouse solutions on Azure Databricks.
You will work closely with data architects, business analysts, analytics teams and business stakeholders to turn enterprise data into reliable, business-ready data products.
The role requires 5, 10 years of overall data engineering experience with solid hands-on work in Azure Databricks, PySpark, SQL and Delta Lake.
Responsibilities
Design, build and maintain scalable data pipelines with Azure Databricks, PySpark, Python, SQL and Delta Lake.
Set up Bronze, Silver and Gold Lakehouse layers and reusable data engineering frameworks.
Build complex ETL and ELT solutions that draw on SAP and other enterprise source systems.
Ingest SAP data and apply the right business and transformation logic to it.
Create data solutions that support manufacturing processes, reporting and analytics.
Handle batch and incremental loading, schema evolution, historical data and pipeline recovery.
Tune complex Spark and PySpark jobs and SQL transformations for scale and speed.
Add data quality checks, validation, reconciliation, monitoring and exception handling.
Model data dimensionally, including Fact/Dimension and Star Schema designs.
Apply good engineering practice across Git, CI/CD, automated deployment, coding standards and documentation.
Turn technical and business requirements from architects and stakeholders into scalable data solutions.
Take part in design and code reviews and help improve Databricks engineering practices.
Requirements
5, 10 years of experience in data engineering, data warehousing or data platform development.
Hands-on experience with Azure Databricks.
Strong skills in PySpark, Apache Spark, Python and SQL.
A good grasp of Delta Lake and Lakehouse architecture.
Experience building Bronze, Silver and Gold (Medallion) architectures.
A track record of building enterprise-scale ETL, ELT and data pipelines.
Experience with data modelling, including Fact/Dimension models and Star Schemas.
Working knowledge of data quality, governance, metadata, lineage and security.
Experience with Git, CI/CD and DevOps for data engineering.
Strong troubleshooting, performance tuning and problem-solving skills.
Good communication skills and the ability to work with technical and business teams spread across the world.
Nice to have: experience with SAP as a source system and a good understanding of SAP data extraction and integration.
Nice to have: experience in oil and gas, energy, manufacturing or customer experience, with an understanding of the data behind those business processes. This is required for the role.
Preferred: Databricks Certified Data Engineer Associate.
A strong advantage: Databricks Certified Data Engineer Professional.
An added advantage: Microsoft Azure or data engineering certifications.
Good to have: Databricks AI Certification.
Conditions
The company is an equal-opportunity employer and welcomes applicants of all backgrounds and genders (m/f/d).