Compute Platform Engineer II

Description

The company is the company's Onyx Research Data Platform group, a large investment by the company R&D and Digital & Tech to make data and prediction more useful for finding new medicines.

The group covers product and portfolio leadership, data engineering, infrastructure and DevOps, data, metadata and knowledge platforms, and AI/ML and analysis platforms.

The Compute Platform Engineering team is building a platform of toolchains and workflows that speed up application development and scale computational experiments. It links computation with project metadata, logs, experiment configuration and performance tracking, and spans cloud and high-performance computing (HPC).

The platform is metadata-focused and CI/CD-driven. It supports the full lifecycle: interactive work in notebooks, large batch jobs, observability and production deployments.

A Compute Platform Engineer II turns vague business or technical problems into clear specifications and delivers them well. They measure both the impact of their work and how their systems run. They set the standard for the team in code quality, documentation, DevOps and testing, keep services robust, and are the escalation point for existing services, pipelines and workflows.

You will know the tools of your specialty and of your users, and take part in the open-source community around them, including contributing pull requests where it makes sense.

Responsibilities

• Design, build and run tools, services and workflows that solve key business problems with high value.
• Build key parts of a hybrid on-premises and cloud compute platform for interactive and scalable batch work, and create processes to move existing HPC users and teams onto it.
• Own code-driven builds of environments, applications and containers/images, and CI/CD-driven application deployments.
• Advise science users on scaling applications to petabytes of data, using a solid grasp of software engineering, algorithms and hardware and how they affect performance.
• Optimise the design and execution of complex solutions in large distributed computing environments.
• Write well-engineered software with automated test suites, technical documentation and an operational strategy.
• Apply platform abstractions consistently so logging and lineage stay high quality and uniform.
• Follow coding best practices, take part in code reviews, and help raise the team's standards.
• Follow the QMS framework and CI/CD best practices, and help improve them.

Requirements

• Bachelor's degree in Data Engineering, Computer Science or Software Engineering.
• Experience with Python.
• Experience with cloud platforms.
• Experience with high-performance computing (HPC).
• Knowledge of at least one common programming language, such as Python, Go, C++, Scala or Java, including tools for documentation, testing and operations or observability.
• Strong skill with modern development tools and practices, such as git/GitHub, DevOps tooling and metrics or monitoring.
• Cloud expertise on AWS, Google Cloud or Azure, including infrastructure-as-code and scalable compute services such as Google Batch and Vertex.
• Experience building CI/CD pipelines with git and a common CI/CD stack such as Azure DevOps, CloudBuild, Jenkins, CircleCI or GitLab.
• Expertise with Docker, Kubernetes and the wider CNCF ecosystem, including deployment tools such as Helm.
• Experience with low-level build tools such as make or CMake, and automated build systems such as spack or easybuild.
• Experience running CI/CD implementations with git and a common CI/CD stack, such as Jenkins, CircleCI, GitLab or Azure DevOps.
• Experience with workflow orchestration tools such as Argo Workflow or Airflow, and scientific workflow tools such as Nextflow, Snakemake, VisTrails or Cromwell.
• Experience tuning and optimising application performance in parallel and distributed computing, using libraries such as MPI, OpenMP or Gloo, with a deep understanding of hardware, networks and storage.
• Proven strength working in agile teams using tools such as Jira and Confluence.
• Familiarity with techniques and optimisations for high-performance applications, and engagement with open-source communities, including contributing to their tools where possible.
• Algorithms, data assessment, decision making, exploratory data analysis (EDA), feature engineering, hypothesis testing, mathematical modelling, predictive modelling, probabilistic modelling, statistical analysis and statistical analysis techniques.

Conditions

The team works in an agile culture, and flexible working can be discussed with the hiring team.

Applicants needing adjustments to the process can contact the UK recruitment adjustments mailbox.

The company does not accept referrals from employment agencies or businesses without prior written authorisation from its procurement or HR department; otherwise it is not liable for any fees.

US licensed healthcare professionals may have interview-related expenses captured and reported under US transparency laws.