Staff Software Engineer - Reporting, Data Platform & Observability

Description

Strength in Trust

OneTrust’s mission is to enable innovation through the responsible use of data and AI. We believe that ensuring data is trusted shouldn’t slow teams down—it should accelerate what’s possible. This led us to develop the first technology platform for responsible data use in 2016. Today, with AI representing the latest and most impactful expansion of data yet, OneTrust is once again redefining what responsible innovation looks like. OneTrust, the AI‑Ready Governance Platform, unifies regulatory intelligence, automation, and connected governance workflows so businesses can continue to move at the speed of AI while ensuring good governance to prevent data misuse at scale. Trusted by thousands of organizations worldwide, OneTrust is shaping the future where trusted data becomes a transformative force for business and society.

The Challenge

OneTrust is seeking a Staff Software Engineer to join the Reporting and Data Platform team. This is a hands-on individual contributor role focused on designing, building, operating, and improving distributed backend services and data-processing platforms.

You will work across Java microservices and Python/PySpark data pipelines, with a strong focus on reliability, scalability, performance, and observability. You will take complex or ambiguous problems from investigation through production delivery and help improve the systems that power reporting and data-driven experiences.

Your Mission

Technical Ownership and Delivery

Own complex features and technical improvements from discovery through production rollout, making substantial hands-on contributions across backend services and data-processing pipelines.

Investigate ambiguous problems, identify root causes, evaluate trade-offs, and implement pragmatic solutions that improve code quality, maintainability, automated testing, and operational readiness.

Review code and technical designs, document important implementation decisions and system behavior, and partner with product managers, engineers, and other teams to clarify requirements and deliver outcomes.

Apply AI-assisted engineering tools such as Devin, Claude, or similar systems to accelerate delivery while maintaining production-quality design, code, tests, security, and operational readiness.

Backend and Distributed Systems

Design and implement production services using Java, Spring Boot, and Maven, including APIs, asynchronous workflows, report generation, aggregation, export, and scheduling capabilities.

Develop event-driven functionality using Kafka and related messaging patterns, and work with caching technologies, relational storage, and service-to-service integrations.

Improve service performance, scalability, fault tolerance, and resource efficiency through appropriate patterns for retries, idempotency, caching, backpressure, concurrency, and failure recovery.

Diagnose issues across services, queues, databases, and downstream dependencies, and modernize established capabilities incrementally while maintaining production stability.

Data Engineering

Build and maintain ingestion and transformation pipelines using Python, PySpark, Azure Databricks, and Delta Lake across batch and streaming workloads.

Implement schema evolution, checkpoint management, deduplication, replay, late-arriving-data handling, and standardized data-layer patterns.

Optimize Spark joins, partitioning, Delta operations, cluster utilization, and query performance while troubleshooting failed, delayed, or inefficient Databricks workloads.

Protect tenant boundaries across joins, aggregations, deduplication, and Delta operations; implement data-quality controls; and monitor data freshness, completeness, and correctness.

Work securely with Azure storage, identities, secrets, and encryption mechanisms.

Observability, On-Call, and Operational Excellence

Improve observability across backend services, event-driven workflows, and data pipelines using meaningful metrics, structured logs, traces, and business telemetry.

Build and maintain actionable dashboards, monitors, and alerts using Datadog and Grafana, applying OpenTelemetry, Prometheus, and Micrometer patterns where appropriate.

Participate in the on-call rotation and incident-response workflows, using PagerDuty, Datadog monitors, or equivalent platforms to diagnose production issues and drive sustainable resolution.

Reduce recurring alerts and operational toil by improving alert quality, eliminating noisy or non-actionable monitors, creating runbooks and diagnostic tools, and implementing corrective actions from blameless incident reviews.

Improve end-to-end correlation and monitor availability, error rates, latency, ingestion lag, data freshness, event throughput, consumer lag, job health, rejected records, checkpoint health, tenant-specific failures, data-quality violations, and Spark resource utilization.

What Success Looks Like

You require limited direction after understanding the desired outcome and relevant constraints, and you break ambiguous problems into concrete, deliverable work.

You own work through design, implementation, testing, deployment, production validation, and ongoing operation.

You use production evidence and telemetry to prioritize improvements and resolve root causes rather than repeatedly treating symptoms.

You reduce alert volume and operational toil over time without hiding genuine system risks, leaving systems easier to operate after each incident.

You make sound trade-offs among delivery speed, reliability, performance, security, cost, and maintainability while collaborating constructively without formal authority.

Requirements

• You are a self-directed, hands-on Staff Engineer who enjoys solving complex problems across distributed services and data platforms. You think in systems and trade-offs, take ownership of production behavior, and use clear design thinking to simplify solutions and reduce code-delivery cycles. You are motivated by building reliable, secure, and maintainable systems and by improving them over time.
• Comfortable working across service, platform, data, and partner-team boundaries without needing formal authority.
• Pragmatic about when to build, reuse, or modernize, with sound judgment around reliability, performance, security, cost, and maintainability.
• Committed to test-driven development, early validation, and production-quality engineering practices.
• Thoughtful about using AI-assisted development tools to accelerate implementation while preserving engineering judgment and accountability.
• Focused on reducing recurring failure modes, alert noise, and operational burden through engineering improvements.
• Your Experience Includes
• Required
• 8+ years of experience building and operating production software systems as a highly autonomous individual contributor.
• Strong proficiency in Java and Spring Boot, with experience designing and operating distributed systems and microservices.
• Production experience with asynchronous or event-driven systems, preferably Apache Kafka.
• Strong experience with Python, PySpark, Apache Spark, and Delta Lake, plus production experience with Azure Databricks or a comparable managed Spark platform.
• Hands-on experience with test-driven development, automated testing strategies, and quality gates that support fast, reliable delivery.
• Strong understanding of metrics, logs, distributed tracing, dashboards, monitoring, and alerting, including hands-on experience with Datadog and Grafana.
• Experience creating or responding to PagerDuty incidents, Datadog alerts, or equivalent production alerting workflows, and willingness to participate in an on-call rotation.
• Experience using AI engineering tools such as Devin, Claude, or similar systems to produce production-ready code, tests, documentation, and operational improvements.
• Strong design-thinking skills and the ability to reduce delivery-cycle time through clear architecture, smaller increments, reusable patterns, and pragmatic technical trade-offs.
• Ability to independently diagnose complex performance and reliability problems and communicate implementation decisions and technical trade-offs clearly.
• Preferred
• Experience with both batch and streaming data pipelines and with optimizing Spark or Databricks workloads for performance, reliability, and cost.
• Experience with Databricks SQL, Databricks SDKs, Delta operations, and schema migrations.
• Familiarity with Azure Blob Storage, Azure Identity, and Azure Key Vault.
• Experience operating reporting, analytics, dashboard, or large-scale export systems.
• Experience with Kubernetes, containers, CI/CD, and infrastructure as code.
• Experience defining or applying service-level indicators, service-level objectives, and error budgets, and using incident and alert trends to prioritize engineering work.
• Understanding of data governance, encryption, audit-ability, and tenant isolation.
• Experience modernizing established production systems incrementally.
• For California, Colorado, Connecticut, Nevada, New York, Rhode Island, and Washington-based candidates: the annual base pay range for this role is listed below. Within this range, individual pay is determined by several factors, including location, job-related skills, work experience, and relevant education and/or training. This role may also be eligible for discretionary bonuses, equity, and/or commissions, as well as benefits.

Conditions

$139,725 - $209,588 USD

Where we Work

We are embracing an office-first culture, encouraging three days a week in office for most roles, with meaningful opportunities to collaborate and celebrate in person.

Each role may have specific requirements or flexibility depending on the scope of the position, so we encourage you to verify this with your recruiter during your first interview.

Benefits

• As an employee at OneTrust , you will be part of the OneTeam . That means you’ll receive support physically, mentally, and emotionally so that you can do your best work both in and out of the office. This includes comprehensive healthcare coverage, flexible PTO, equity RSUs, annual performance bonus opportunities, retirement account support, 14+ weeks of paid parental leave, career development opportunities, company-paid privacy certification exam fees, and much more. Specific benefits differ by country. For more information, talk to your recruiter or visit onetrust.com/careers.
• Resources
• Check out the following to learn more about OneTrust and its people:
• OneTrust Careers on YouTube
• @LifeatOneTrust on Instagram
• Your Data
• You have the right to have your personal data updated or removed. You also have the right to have a copy of the information OneTrust holds about you. Further details about these rights are available on the website in our Privacy Overview . You can change your mind at any time and have your personal data removed from our database. In order to do this you must contact us and let us know you wish to be removed. The request should be made on the Data Subject Request Form .
• Recruitment fraud warning: OneTrust is aware of scams involving false offers of employment with our company. The fraudulent jobs, interviews and job offers use fake websites, email addresses, group chat and text messages. Be aware that we never ask candidates for personal information, IDs or bank information during the interview process. We do not interview prospective candidates via instant message or group chat, and do not require candidates to purchase products or services, or process payments on our behalf as a condition of any employment offer. Please note that any legitimate interview availability requests will come directly from a OneTrust recruiter with an "@onetrust.com" email address. You may also receive legitimate emails from "@us.greenhouse-mail.io". Recruiters will only reach out to candidates who have applied for a role through our ATS (Greenhouse) or prospects via LinkedIn InMail. Job offers will come from a recruiter and may have a "@docusign.net" email address. For more information or if you have been targeted please reach out to askrecruiting@onetrust.com .
• Our Commitment to You
• When you join OneTrust you are stepping onto a launching pad — the countdown has begun. The destination? A career without boundaries working alongside a diverse and inclusive crew who is passionate about doing meaningful work. As a pioneer, your voice and expertise will help chart the direction of an entirely new category. Our commitment to putting people first starts with you. Your growth is part of the mission. Our goal is to give you the power to embark on the next phase of your uniquely, unique career.