The company runs a platform that evaluates how AI models perform on real work. It was founded by researchers from UC Berkeley's SkyLab.
Tens of millions of people use the platform each month. Their preferences feed evaluations that leading AI labs, enterprises, and independent researchers use, along with open datasets covering agentic coding, creative generation, and professional productivity.
The team includes researchers, academics, builders, and creatives from UC Berkeley, Google, Stanford, and DeepMind. People are valued for craftsmanship, curiosity, and impact more than for seniority.
The company's cloud runs on managed platforms for hosting, data, workflows, source control, and builds. This role sets the security requirements across those platforms and builds the controls into the infrastructure.
Responsibilities
Design the cloud identity model for workloads. Apply least-privilege IAM, workload identity federation, and short-lived credentials, and decide what each workload can do and how it proves its identity.
Own every machine secret from start to finish: a named owner, storage in a central secret manager, and a rotation path that has been tested so rotation is routine and causes no product downtime.
Define what each environment and service may reach, including outbound connection limits, separation between environments, and boundaries around agent runtimes.
Write security detections and alerts for risky changes to cloud infrastructure and build systems, plus response steps that another engineer can follow alone.
Put cloud security checks into infrastructure code and build pipelines so the secure option is also the simplest.
Secure the machine credentials and permissions used by source control, build pipelines, and cloud deployments.
Evaluate the technical access that vendors and managed platforms ask for before onboarding them.
Lead the technical response to cloud infrastructure security incidents, working with affected teams through recovery.
Keep cloud security posture visible to engineering and leadership, and keep improving it.
Take a share of the security on-call rotation.
Work hands-on with platform, infrastructure, and product engineers, doing the work yourself rather than directing it from a distance.
Requirements
6+ years in security or infrastructure engineering, with at least 3+ years focused on cloud security in production.
Deep, current hands-on experience securing AWS, GCP, and Cloudflare, with concrete configuration work you can describe in detail from the last 3 months.
You have owned cloud security in a previous role, not only worked through findings. You have made and defended decisions about IAM and secrets, lived with their costs, and can name a call you got wrong and changed.
Strong working knowledge of workload identity: OIDC, workload identity federation, IAM design, and secret managers. You can explain how a workload proves its identity without a stored key.
You have rotated production database or API credentials without an outage and can say what you would change next time.
You have written cloud detections, been paged by them, and investigated a real incident using cloud audit logs.
Working knowledge of Terraform or a similar infrastructure-as-code tool, and of at least one automation language such as Python, Go, or TypeScript.
Experience securing CI/CD pipelines and build systems, such as GitHub Actions and deployment credentials.
You use AI tools in your own work to build, script, and investigate, and you know where they fit in a security workflow and where they do not.
You can separate real exposure from theoretical risk and state which is which.
You are comfortable improving existing systems and documentation while keeping work on track.
Bonus: experience securing serverless platforms such as Vercel.
Bonus: experience securing infrastructure for AI agent runtimes or sandboxed code execution.
Bonus: experience with Kubernetes security, Postgres access controls, or data platform security such as Databricks.
Bonus: experience leading incident response in a cloud environment.
Bonus: experience with compliance evidence for cloud controls, such as SOC 2 or ISO 27001.
Bonus: experience as an early security hire at a fast-growing startup.
Bonus: experience handling data protection requirements in logs and access controls.
Conditions
This is a staff-level role based in the San Francisco office, onsite.
The role reports to the Head of Security.
Pay is competitive for the market where the employee is based. The range is listed on each posting and depends on experience and work location.
Benefits
Comprehensive health and wellness coverage, including medical, dental, and vision, plus additional support programs.
Work on cutting-edge AI alongside a small, mission-driven team.
A culture that values transparency, trust, and community impact.