You and the process

With a person
Anna, in-house recruiter
9 years hiring engineers
30 minutes with a real recruiter
They read your CV with you, on a call, and say where the offers are being lost.
Didn’t find what you were looking for? Tell us what to build
AG Coders

Lead Data Platform Engineer (Databricks & Data Governance)

  • Remote
  • 6+ years

Salary

$104,000 - 128,000/ year

AI summary

For members

The whole posting in a few lines. Sign up to read it here and on every role you open.

Sign Up to Read

Description

The company needs a lead to drive the design, build and rollout of Test Data Management and Data Governance frameworks on Databricks.

The role centres on automated pipelines for synthetic data and data masking, company-wide security controls, and self-service access to compliant datasets for engineering and QA staff.

Location: United States, remote or hybrid, flexible as needed. Full-time.

Responsibilities

  • Lead the adoption of Test Data Management frameworks across several projects on Databricks.
  • Set tight governance over sensitive data in non-production environments, covering role-based access control, metadata management, data classification and lineage.
  • Design and build automated, scalable pipelines for ingestion, transformation, test data generation, masking and environment refreshes.
  • Plan synthetic data approaches for lower environments and build reusable provisioning frameworks so teams can self-serve compliant test data.
  • Work closely with QA, application development, data governance and information security teams to enable the platform and set shared practices.

Requirements

  • 8+ years of progressively responsible experience in data engineering, data architecture or data platform roles.
  • 3+ years of hands-on work with Databricks, including Lakehouse, Delta Lake, Unity Catalog and PySpark.
  • Proven record of implementing Medallion architecture (Bronze, Silver, Gold) and rolling out Unity Catalog or enterprise data governance.
  • Strong software engineering skills in Python and PySpark.
  • Hands-on experience with Azure (ADF, Azure DevOps, Synapse) and/or AWS (S3, Lambda, Glue, Redshift, MWAA).
  • Preferred, highly desired: hands-on Test Data Management work, including data masking, de-identification and synthetic data generation.
  • Preferred, highly desired: familiarity with AI-powered data discovery and classification and generative AI tools such as Databricks Mosaic AI.
  • Preferred, highly desired: experience building data observability and monitoring frameworks.
  • Preferred, highly desired: a track record of rolling out cross-team frameworks and enabling teams technically.
  • Preferred, highly desired: proficiency with CI/CD pipelines and Infrastructure as Code using Terraform, Azure DevOps or GitHub Actions.
  • Preferred, highly desired: working knowledge of regulations covering sensitive data, including PII, GDPR, CCPA and HIPAA.

Conditions

Engagement as an independent contractor, 1099 or Corp-to-Corp.

Initial contract of 1 year, with strong potential for extension based on performance and project needs.

Full-time, remote or hybrid in the United States, adjusted as needed.

Equal opportunity statement omitted.

Benefits

  • Pay of up to $60 USD per hour, set by a technical assessment and your level of experience.
  • Chance to design enterprise-level data governance and AI-ready frameworks from the ground up.

Where you’d work

Fully remote

You can work from

  • United States

About the company

AG Coders

Your chances

We've checked whether it's still hiring and how crowded it is.

  • 20 checks run
  • 0 red flags

Still hiring?

13 checks

No red flags

How crowded?

7 checks

No red flags

Fits Me

How well does this role fit you?

Answer a few questions or drop your CV, and every role gets a fit score with the reasons, this one first.

  • Your field
  • Level
  • Stack
  • Work model
  • Salary floor
  • Must-haves
Details11 facts · Role, Location, Compensation, Employment
Tech stack
  • Python
  • AWS
  • Azure
  • Spark
  • Databricks
  • LLMs
Seniority
Lead
Type
Contract
Specialty
Data Engineering
Region
United States
Pay period
Annual
Show 5 more factsShow less

Role

Category
Data & Analytics
Specialty
Data Engineering
Seniority
Lead
Experience
8+ years
Tech stack
  • Python
  • AWS
  • Azure
  • Spark
  • Databricks
  • LLMs

Location

Work model
Remote
Region
United States
Remote from
  • United States

Compensation

Salary
$104,000 - 128,000 / year
Pay period
Annual

Employment

Type
Contract

Something wrong with this vacancy?

Similar vacancies

Share this vacancy

What's wrong with it?

The employer never sees who reported.

Reason