Senior Data Engineer with Python and Databricks

Description

The company is a consultancy that builds digital platforms and experiences for clients across many industries, working with leading brands.

The role sits on a data team working with data scientists and analysts to deliver data pipelines.

Responsibilities

• Gather data needs with data scientists and analysts, then build pipelines that meet them.
• Design, build and maintain cloud-based pipelines for ingesting, processing and transforming data.
• Build and run data solutions in Azure using any of Data Factory, Synapse, Databricks or Spark, Polars or Pandas, and Fabric.
• Add data validation and cleansing steps to keep data accurate, complete and reliable.
• Make pipelines more scalable, efficient and cost-effective.
• Monitor pipelines, resolve problems quickly, and keep data consistent and available.

Requirements

• Strong Python and SQL skills.
• Understanding of data design principles such as Kimball and star schema, plus data warehousing and data modelling.
• Solid experience with relational databases; NoSQL experience is ideal.
• Strong grasp of ETL and ELT processes.
• Exposure to data monitoring and observability tools.
• Relevant experience with event-driven architecture.
• Good understanding of Agile ways of working.

Conditions

Hybrid work with flexible working hours.

Equal opportunity: hiring is based on merit, skills, qualifications and potential. Tell the company if you need adjustments during recruitment.

Benefits

• Competitive salary package, share plan, company performance bonuses, value-based recognition awards and a referral bonus.
• Career coaching, global career opportunities, non-linear career paths and internal development programmes for management and technical leadership.
• Complex projects, rotations, internal tech communities, training, certifications, coaching, online learning platform subscriptions, pass-it-on sessions, workshops and conferences.
• Employee assistance programme.
• Global internal wellbeing programme and access to wellbeing apps.
• Global internal tech communities, hobby clubs and interest groups, inclusion and diversity programmes, and events and celebrations.