Data Engineer / Databricks Platform Administrator

Description

The team is building a secure, scalable, enterprise-grade Databricks platform on AWS, and this hybrid role mixes platform engineering and administration with hands-on data engineering delivery.

You will help set up and run the platform while also taking part in data pipeline work, governance efforts and engineering practice.

Responsibilities

• Build Databricks workspaces on AWS from scratch.
• Set up and manage Unity Catalog, including metastores, catalogs, schemas, storage credentials and access controls.
• Design secure network architecture on AWS using VPCs, Private Link, network isolation and enterprise security practices.
• Create and maintain IAM roles, instance profiles and cloud access models.
• Write and upkeep Infrastructure-as-Code with Terraform or similar tooling.
• Build CI/CD pipelines covering platform configuration, deployment and governance controls.
• Define and enforce cluster policies, workspace standards and operating guardrails.

Requirements

• Hands-on experience administering Databricks environments.
• Experience implementing and managing Unity Catalog and Databricks governance frameworks.
• Experience building and supporting production data pipelines.
• Strong Python and SQL development skills.
• Strong AWS experience, including VPC, Private Link, IAM, S3, networking and security architecture.
• Strong Infrastructure-as-Code experience with Terraform (preferred) or equivalent tools.
• Experience setting up CI/CD pipelines for infrastructure and platform deployments.
• Australian citizenship and an active security clearance are required, because the work involves sensitive information.

Conditions

Hybrid work arrangement.

Australian citizens with an active security clearance only.

Benefits

• Competitive salary package.
• Flexible working options, including remote and hybrid models.
• Training programmes, certifications and mentorship to build your skills.
• Defined career paths and work on high-impact projects for global clients.
• Health insurance, wellness programmes and mental health support.
• Generous leave, including parental leave and extra paid holidays.
• An inclusive workplace that values diversity.