The team is building data pipelines and Lakehouse solutions and needs an engineer with 5-10 years of data engineering experience, with solid hands-on work in Azure Databricks, PySpark, SQL and Delta Lake.
You will work closely with data architects, business analysts, analytics teams and business stakeholders to turn enterprise data into reliable data products that the business can use.
Responsibilities
Build and maintain scalable data pipelines using Azure Databricks, PySpark, Python, SQL and Delta Lake.
Set up Bronze, Silver and Gold Lakehouse layers and reusable data engineering frameworks.
Create complex ETL and ELT solutions that draw on SAP and other enterprise source systems.
Extract and transform data from SAP environments, applying the right business and transformation logic.
Build data solutions that support manufacturing processes, reporting and analytics.
Handle batch and incremental loading, schema evolution, historical data and pipeline recovery.
Tune complex Spark and PySpark workloads and SQL transformations for scale and speed.
Add data quality checks, validation, reconciliation, monitoring and exception handling.
Design analytical and dimensional models, including Fact/Dimension and Star Schema models.
Apply sound engineering practice in Git, CI/CD, automated deployment, coding standards and documentation.
Work with data architects and business stakeholders to turn technical and business needs into data solutions.
Take part in design reviews, code reviews and ongoing improvement of Databricks engineering practice.
Requirements
Essential: 5-10 years in data engineering, data warehousing or data platform development.
Strong hands-on experience with Azure Databricks.
Strong skills in PySpark, Apache Spark, Python and SQL.
Good understanding of Delta Lake and Lakehouse architecture.
Experience building Bronze, Silver and Gold (Medallion) architecture.
Strong record of building enterprise-scale ETL, ELT and data pipelines.
Experience with data modelling, including Fact/Dimension models and Star Schemas.
Understanding of data quality, governance, metadata, lineage and security.
Experience with Git, CI/CD and DevOps practices for data engineering.
Strong troubleshooting, performance tuning and problem-solving skills.
Good communication skills and the ability to work with technical and business teams spread across the world.
Nice to have: experience with SAP as a source system and a good grasp of SAP data extraction and integration.
Oil and gas, energy, manufacturing or customer experience background is required, with an understanding of the data tied to business processes.
Certifications: Databricks Certified Data Engineer Associate is preferred.
Databricks Certified Data Engineer Professional would be a strong advantage.
Microsoft Azure or data engineering certifications are an added advantage.
Databricks AI certification is good to have.
Conditions
The company is an equal-opportunity employer and welcomes applicants of every background and gender (m/f/d).