The company is hiring a senior Databricks data engineer with 5 to 10 years of data engineering experience, with hands-on work in Azure Databricks, PySpark, SQL and Delta Lake.
The role builds scalable data pipelines and Lakehouse solutions. You will work closely with data architects, business analysts, analytics teams and business stakeholders to turn enterprise data into reliable data products that the business can use.
Responsibilities
Build and maintain scalable data pipelines with Azure Databricks, PySpark, Python, SQL and Delta Lake.
Set up Bronze, Silver and Gold Lakehouse layers and reusable data engineering frameworks.
Build complex ETL and ELT solutions that draw on SAP and other enterprise source systems.
Ingest SAP data and apply the right business and transformation logic to it.
Build data solutions for manufacturing processes, reporting and analytics.
Handle batch and incremental loading, schema evolution, historical data and pipeline recovery.
Tune complex Spark, PySpark and SQL workloads for scale and speed.
Put in place data quality checks, validation, reconciliation, monitoring and exception handling.
Design analytical and dimensional models, including Fact/Dimension and Star Schema models.
Apply engineering practices for Git, CI/CD, automated deployment, coding standards and documentation.
Turn technical and business requirements into scalable data solutions with data architects and business stakeholders.
Take part in design and code reviews and help improve Databricks engineering practices.
Requirements
5 to 10 years of experience in data engineering, data warehousing or data platform development.
Solid hands-on experience with Azure Databricks.
Strong skills in PySpark, Apache Spark, Python and SQL.
Good understanding of Delta Lake and Lakehouse architecture.
Experience building Bronze, Silver and Gold (Medallion) architectures.
Proven track record building enterprise-scale ETL, ELT and data pipelines.
Experience with data modelling, including Fact/Dimension models and Star Schemas.
Working knowledge of data quality, governance, metadata, lineage and security.
Experience with Git, CI/CD and DevOps for data engineering.
Strong troubleshooting, performance tuning and problem-solving skills.
Good communication skills and the ability to work with globally distributed technical and business teams.
Nice to have: experience with SAP as a source system and a good understanding of SAP data extraction and integration.
Nice to have: Oil & Gas, energy, manufacturing or customer experience background, with an understanding of the data linked to business processes.
Databricks Certified Data Engineer Associate is preferred.
Databricks Certified Data Engineer Professional would be a strong advantage.
Microsoft Azure or data engineering certifications are an added advantage.
Databricks AI certification is good to have.
Conditions
The company is an equal-opportunity employer and welcomes applicants of all backgrounds and genders (m/f/d).