Senior Big Data Engineer

Description

You will design, build and support data engineering solutions largely on your own, starting with learning the data the company handles through profiling.

You will own solution design, work with the wider engineering group to deliver on time, and keep the platform running reliably in production.

The platform solves healthcare data problems, such as mapping varied drug and diagnosis records to standard coding frames, and loads and processes millions of electronic medical records.

The Systems Engineering team sits within Global Infrastructure & Operations and is one of the company's fastest-growing groups. It works in Agile, with DevOps and test-driven development, and team members have autonomy to organize their own work.

The company provides clinical research services, commercial insights and healthcare intelligence to the life sciences and healthcare industries. Your team works with patient-level data, for uses ranging from retrospective clinical studies to disease progression modelling.

Day to day, you will work with architects who shape solutions and with the quality assurance team.

The platform processes over a billion records at a time. You will build ETL pipelines that acquire, process and load medical data into Databricks.

Responsibilities

• Guide and mentor junior and mid-level engineers, giving technical direction that supports their growth.
• Assign tasks clearly, with expectations and delivery goals kept aligned.
• Own your assigned workstreams, taking responsibility for quality, timing and accountability across the team.
• Build pipelines that handle large volumes of medical data.
• Take part in analysis, system design, development, deployment and maintenance.
• Run peer code reviews.
• Help define and adopt technical standards.
• Write thorough automated unit and integration tests.

Requirements

• 6 years of hands-on work with SQL, Python and Scala.
• 6 years of experience in data engineering.
• Solid understanding of the Databricks platform.
• Experience with big data batch processing and Azure cloud services.
• Working knowledge of the Apache Spark ecosystem, including Delta and Parquet formats.
• Experience designing and implementing complex ETL.
• Experience across data analysis, profiling, integration, quality, warehousing and reporting.
• Experience with CI/CD automation using tools such as Jenkins or GitLab.
• Practical use of AI productivity tools such as Copilot or Cursor in daily development, for code generation, documentation, task automation or debugging.
• Experience with database design and modelling.
• Knowledge of algorithms and design patterns, and how to apply them well.
• Experience processing data volumes in the TB range.
• Experience working in Agile teams.
• Strong analytical and troubleshooting skills.
• Good collaboration skills, with strong written and verbal communication.
• Familiarity with Agile and Scrum project management concepts.
• Ability to explain complex results clearly to both technical and non-technical audiences.
• Experience working with teams in Europe, including cross-cultural communication and remote delivery.
• A strong team player.
• Practical experience building production-grade APIs with FastAPI is a plus.
• Previous healthcare or pharmaceutical data experience would be an advantage.

Conditions

Annualized base pay range of zł188,700.00 to zł350,200.00.

The actual base pay offered may vary based on qualifications such as knowledge, skills, education and experience, as well as location and schedule (full or part time).

Incentive plans, bonuses or other forms of compensation may be offered depending on the position, along with health and welfare benefits.

Benefits

• The company has offices in over 100 countries and is listed on the American stock exchange (NYSE).
• Work with multi-terabyte databases.
• Quarterly work planning, with goals and progress reviewed every 90 days.
• Technical training.
• Possibility to attend technical conferences within the country.
• Favourable employment conditions and benefits.