Lead Data Engineer

Description

You will design and build data processing and data storage components for systems that handle large volumes of data.

You will work in agile teams, giving technical leadership to engineers and owning the quality of the codebase and how well it meets user needs.

The company is a high-growth IT services firm delivering digital technology and agile software development to enterprise customers. With 30 years of work behind it, it has delivered transformational projects for government, the NHS and private sector clients.

Responsibilities

• Own the development of whole components or subsystems within a team, covering design, coding, testing and fixing defects.
• Design and build hands-on, using open source and commercial platforms.
• Set and enforce development best practice, and coach team members to keep the work consistent.
• Work with project architects to take responsibility for non-functional needs of ETL and ELT data pipelines, such as robustness and performance.
• Set the standards for unit and integration testing and make sure the team follows them.
• Own due diligence and integration for software products.
• Lead troubleshooting and performance tuning.
• Work with Operations teams so the application software is ready to run in production.
• Work with security architects and accreditors to meet relevant legal and security requirements.
• Advise customers, managers and team members on the estimated effort and technical impact of user stories and user journeys.
• Contribute technical input to proposals during the sales process.
• Manage, coach and develop a small group of staff, including their performance and career growth. Give your team direction as you solve problems together.

Requirements

• Essential: experience leading a team of engineers building data-intensive system components.
• Essential: experience applying standards for design patterns, coding style guides, and operational readiness such as automation and deployment.
• Essential: solid software development skills in one of Java, Scala or Python.
• Essential: software development experience with data processing platforms from vendors such as Informatica, Azure Databricks, or other relevant ETL tools.
• Essential: expert-level SQL, including SQL extensions for analytical work.
• Essential: expert understanding of distributed data stores and data processing frameworks.
• Essential: the ability to explain technical designs clearly, in writing and out loud.
• Essential: proficiency in designing analytical and operational data models.
• Essential: a keen interest in AI technologies.
• Desirable: comfort with data warehouse methods and techniques.
• Desirable: a habit of sharing your views on data practices.
• Desirable: certification in data services on AWS, Azure or GCP.
• Desirable: skill in continuous improvement and in contributing to data best practice.
• Desirable: participation in development or technology communities.
• Desirable: practical experience with AI technologies, tools, processes and delivery.
• Expected behaviours: determination when overcoming obstacles, honesty and respect in how you treat others, creativity in solving complex problems, and cooperation through sharing information, knowledge and experience.

Conditions

Accommodations and adjustments are available on request, and the team will tailor the recruitment process to your needs in a private conversation.

The company welcomes applicants of all backgrounds and treats every candidate on equal terms.