The AI and Automation department works with large, complicated datasets and builds scalable tools that support regulatory requirements and make processes run more efficiently. The team works across a wide technology stack, builds the infrastructure beneath it, and uses cloud services and machine learning to keep data accurate and improve performance on the company's global platforms.
You will help create and run pipelines that move data from the data lake into a new cloud-based Enterprise Data Warehouse (EDW). The core architecture and data models are already in place, so your focus will be building and improving the pipelines that load fact and dimension tables, plus helping with automation and data quality.
In-Play betting is central to the business, with over 1.38 million In-Play sporting events a year, more than 750 concurrent fixtures at peak, and 750,000 live sports streams, more than anyone else in Europe. The platform handles over 6 million HTTP requests a day and more than 1.5 million bets per hour at peak.
Responsibilities
Build, maintain and improve reliable pipelines that bring data from the data lake into the cloud EDW.
Build and support the loading of fact and dimension tables, following the established architecture, data models and development standards.
Write efficient SQL and transformation logic, applying good practice for data quality, testing, performance, monitoring, error handling and reliability.
Work in a shared, source-controlled repository with other developers, taking part in peer reviews, CI/CD and controlled deployment of data solutions.
Find and resolve pipeline and data quality issues, and help with automation, technical documentation and ongoing EDW improvements.
Work with engineers, analysts and stakeholders in an agile way, using AI tools responsibly for development, troubleshooting, documentation and solution design.
Requirements
Strong SQL skills, including writing efficient, maintainable queries for analytics and data warehouse work.
Hands-on experience with cloud data platforms, especially Google BigQuery, and with building, maintaining and troubleshooting pipelines in Apache Airflow or similar tools.
Proven ability to load, transform and model data for cloud data warehouses, including fact and dimension tables. You have some exposure to dbt or Dataform.
Solid grasp of data engineering basics: data quality, testing, monitoring, performance and reliability.
Experience using Git for source control and working in a shared repository with several developers.
Working knowledge of CI/CD, peer review and development standards for controlled testing and deployment.
Background working in agile teams, contributing to technical discussions, planning and delivery.
Ability to use AI tools responsibly for development tasks such as code generation, refinement, troubleshooting and documentation.