The company builds an AI Site Reliability Engineer for large enterprises. Its agent autonomously reads and reasons across very large production environments to diagnose, fix and prevent incidents, which matters more as AI-generated code adds to production complexity.
The company already runs in mission-critical settings at large enterprises such as PepsiCo, Nubank, Harvey and DigitalOcean. Its roots are in AI research, and its team combines researchers from MIT, Harvard, Berkeley, Columbia and Cornell with engineers and operators from Google, Meta, Datadog, ServiceNow and Citadel Securities.
The role is a Principal Engineer position on the Data Platform team, based in New York.
Responsibilities
Own the technical direction of the data platform, which is the foundation the agent's investigations depend on. It is a petabyte-scale system that takes hundreds of billions of log lines a day from one customer, at hundreds of megabytes per second, and makes them queryable within minutes.
Set the long-term architecture for four data paths: direct push ingestion, scheduled pulls from third-party observability platforms, non-telemetry data, and transformation pipelines. Specify the interfaces, invariants and evolution path so they scale together.
Choose the storage and query design for 10-100x today's scale, weighing time-series, columnar/OLAP and lakehouse options, along with retention, cost and performance targets for the agent's key queries.
Treat the agent as a core workload. Set performance targets for its tool queries, freshness guarantees for its data, and safe sandboxing for the ad-hoc queries an LLM will write.
Work with the AI Research team to supply investigation history, evaluation datasets and derived signals for the agent and its auto-research loops.
Build the orchestration and transformation layer: scheduled, idempotent, observable jobs that bring in non-telemetry context and produce materialized views, rollups, entity graphs, and LLM-based summaries and enrichment cached before investigations begin.
Define and meet SLOs for availability, data freshness and query latency. Ship systems with monitoring, runbooks and safe rollouts, and lead the technical response when failures occur.
Design redaction, tenant isolation and data residency controls that meet the security and compliance standards of large banks, payment companies and tech firms.
Mentor senior engineers and set standards and reference implementations that other teams adopt.
Stay hands-on: write the design docs, write the code that tests them, and join architecture reviews and Fortune 100 customer deployments. Bring what you learn from customers and go-to-market teams back into the roadmap.
Work directly with engineering and company leadership on the roadmap and its business impact.
Requirements
Deep distributed-systems knowledge across streaming data, analytical storage and query engines, such as Kafka or equivalent, ClickHouse, Doris, Druid or Pinot, and lakehouse technologies, plus experience building reliable multi-tenant systems.
Strong systems programming in Rust, the platform's main language, or comparable Go experience, and comfort with Python.
Experience deploying on Kubernetes, Helm, Terraform and AWS, ideally in environments you do not fully control, such as BYOC, on-prem or regulated customer clouds.
A record of leading multi-quarter migrations through adoption and deprecation, not just design.
You write production code regularly and can walk through recent code you personally shipped.
Sound cost judgment, weighing performance, cost, durability and simplicity with real numbers, and knowing what your systems cost to run.
Clear technical writing, the ability to make firm tradeoff decisions, build alignment without formal authority, and hold your own with enterprise customers in architecture reviews.
Experience building the backend of an observability, logging, APM or SIEM product.
Fluency with OpenTelemetry and OTLP, and with the data models and APIs of major observability vendors such as Datadog, Splunk, Elastic, Prometheus and Dynatrace.
Experience with workflow orchestration and pipeline tools such as Airflow, Dagster or Temporal.
Experience building LLM-powered pipelines for enrichment, summarization and embeddings, and controlling their cost.
Background in data governance, including PII redaction, data residency and regulated industries such as financial services.
Contributions to open-source data infrastructure.
Conditions
Full-time, in-person role in New York, near Madison Square Park, five days a week.
The U.S. base salary range is $275,000 to $350,000 USD, plus equity and benefits. Pay depends on location, level and role, and individual offers reflect experience, skills and job-related knowledge.
The company is fully in-office, with a collaborative culture focused on AI-powered software maintenance.
Benefits
Startup equity, health insurance, fertility benefits, a technology setup stipend and flexible time off.
In-office snacks, team happy hours and outings, an annual company offsite, and built-in time to collaborate across teams.
Where you’d work
From the office, 4 or more days a week in the office