Senior Data Engineer

Description

The company is an experience innovation firm that works with large, well-known brands. Teams work on digital solutions, customer experiences and data-driven transformation across industries.

The role sits in a global team with a culture of learning, autonomy and creative work, and collaboration spans borders.

Responsibilities

• Design, build and tune modern cloud data platforms that feed analytics, AI and data products.
• Look past the technical brief to understand the business problem behind the data, and use analysis and experience to help shape requirements rather than only building what was specified.
• Build batch, streaming and near-real-time pipelines that deliver curated datasets with solid governance, security and observability.
• Support AI and GenAI work by building pipelines for machine learning, causal modeling, and LLM applications such as RAG and agent-based systems, working alongside data scientists, ML engineers and platform teams.
• Work mainly on AWS with Databricks, with growing GCP work including cloud migrations, and some Azure or Fabric projects. You will be placed on projects that match your strongest platform, with chances to widen your skills.
• Design and implement scalable platforms and pipelines, mainly on AWS and Databricks, and on GCP or Azure/Fabric where projects need it.
• Develop reliable batch, streaming and near-real-time pipelines with Spark and Delta Lake.
• Build ingestion, transformation and curation workflows for structured and unstructured data.
• Apply lakehouse patterns and bronze, silver, gold layering to create reusable, scalable systems that fit enterprise needs.
• Help move pipelines and data models between clouds.
• Produce model-ready, production-grade datasets for analytics, ML, causal modeling and optimization.
• Build data flows for GenAI use cases, including LLMs, RAG and vector-based data, and for agent-based architectures.
• Use Databricks MLflow and Unity Catalog, or equivalent tools on other platforms.
• Design logical and physical data models for analytical and operational needs.
• Orchestrate workflows with Databricks Workflows, Airflow, dbt or equivalents such as Azure Data Factory or Cloud Composer, focusing on automation, reliability and maintainability.
• Apply event-driven and streaming architecture patterns.
• Manage governance, lineage, quality and access control (RBAC/ABAC) using Unity Catalog, AWS Lake Formation or equivalents such as Microsoft Purview or GCP Dataplex.
• Monitor data freshness, pipeline reliability and SLA adherence.
• Build serving layers, such as APIs, feature inputs and analytical endpoints, for downstream ML and AI platforms.
• Keep improving pipelines and infrastructure for performance, scalability and cost.

Requirements

• Ask probing questions to understand why stakeholders need something, not only what they want.
• Point out gaps and better options instead of delivering a brief exactly as written.
• Work closely with data scientists, ML engineers, analysts and business stakeholders.
• Help teams adopt data products and contribute to best practices.
• Strong hands-on experience with Apache Spark and Delta Lake.
• Strong Python and SQL.
• Proven experience building batch and streaming pipelines and production-grade data platforms.
• Solid understanding of data modeling, data quality and governance.
• Strong, demonstrable hands-on experience with the Databricks Lakehouse Platform, ideally on AWS with services such as S3, Glue, EMR, Redshift, Kinesis or Lambda.
• Deep expertise in at least one major cloud (AWS, GCP or Azure/Fabric) and a demonstrable ability to work across clouds. Strong Azure/Fabric candidates, with experience in Data Factory, Synapse, Fabric Lakehouse or Azure Databricks, are encouraged to apply if they are willing to work on other clouds.
• GCP experience with BigQuery, Dataflow, Pub/Sub, Cloud Storage or Dataproc is highly valued, especially for migrations.
• Experience with lakehouse architectures and distributed systems.
• Strong understanding of scalability, reliability and performance.
• Naturally curious, with strong problem-solving skills and a collaborative style in cross-functional teams.
• Willing to challenge business requirements rather than accept them at face value.
• Adaptable and keen to work across platforms. Agile or consulting experience is a plus.
• Optional: experience across more than one cloud, or cloud migration experience such as Azure to GCP.
• Optional: Snowflake experience.
• Optional: GenAI and AI data systems, including RAG pipelines, vector databases and LLM data preparation.
• Optional: CI/CD for data pipelines and infrastructure-as-code with Terraform or CloudFormation.
• Optional: streaming technologies such as Kafka, Spark optimization, or ML and experimentation platforms.
• Optional: experience building data products or large-scale analytics platforms.

Conditions

Based in Portugal.

Remote and hybrid options are available, depending on the country.

The company does not require age, gender, marital status or a photo, and reviews candidates on skills, experience and potential.

Reasonable accommodations are available during interviews on request.

Equal opportunity: the company works to include people of all backgrounds.

Benefits

• A competitive compensation package.
• International mobility and professional development programs.
• Learning and development, with access to modern tools, training and industry experts.
• Benefits vary by location, and your recruiter will give full details during hiring.