Data Engineer

Description

The team is hiring a mid-level data engineer to design, tune and keep data pipelines running, with a focus on scale, reliability and performance.

You will also work with data science teams to bring machine learning models into production and run them there.

The company is a European digital services provider with 28,000 consultants and specialists, serving business clients, public sector bodies and society.

Responsibilities

• Build and maintain data pipelines that scale.
• Improve how data processing runs on very large datasets.
• Set up and manage ingestion from several data sources.
• Keep data accurate, dependable and fast across all platforms.
• Work with data science and engineering teams to deploy and run ML solutions.
• Help raise engineering standards and drive ongoing improvements.

Requirements

• You have worked in data engineering for 3+ years.
• You write advanced Python and SQL.
• You can design solutions that are sturdy, easy to maintain, highly scalable and well tuned for large data workloads.
• You have hands-on work with in-memory processing libraries such as Polars or an equivalent.
• You have solid experience with distributed processing frameworks, PySpark in particular.
• You have built ingestion pipelines that pull from REST APIs.
• You apply strong software development practices.
• You use Git, CI/CD pipelines and software testing methods.
• You have deployed, monitored and maintained machine learning pipelines in production.
• Experience with modern data platforms such as Databricks is required.
• Experience orchestrating workflows with a tool such as Apache Airflow is required.
• You know at least one cloud platform: AWS, Azure or GCP.