You and the process

With a person
Anna, in-house recruiter
9 years hiring engineers
30 minutes with a real recruiter
They read your CV with you, on a call, and say where the offers are being lost.
Didn’t find what you were looking for? Tell us what to build
Early Window: Closes in
Company hidden

Protection Scientist Engineer

  • Hybrid

Salary

$220,000 - 425,000/ year

AI summary

For members

The whole posting in a few lines. Sign up to read it here and on every role you open.

Sign Up to Read

Description

The Integrity team builds the scaled systems that find and respond to bad actors and harm on the company's platforms. The work covers frontier harms such as biological weapons and cybersecurity, as well as serious abuses including child safety, nation-state misuse and violence.

Protection Scientists in Integrity design, run and mature defences against harmful uses of the company's products. The role sits between investigation, data science, machine learning, policy and operations, and the value lies in connecting those disciplines to build working defences.

The San Francisco team is hiring for protection work on sextortion, trafficking, national security (including nation-state misuse) and jailbreaks. Depending on your background, you might work on AI-enabled coercion and exploitation, misuse that supports trafficking, coordinated activity by state-linked actors, or attempts to get around model safeguards. You will turn these threats into detection systems and interventions that cut harm.

The role is based in the San Francisco office and includes an on-call rotation for urgent escalations outside normal hours. Some investigations involve sensitive material, including sexual, violent or otherwise disturbing content.

Responsibilities

  • Set the strategy for a harm area or product launch. Turn unclear risks into goals, priorities and roadmaps, and decide trade-offs based on evidence, risk and capacity.
  • Design, build and maintain production harm detection: signals, classifiers and pipelines written in SQL and Python, using machine learning and language models, and connected to shared engineering platforms.
  • Work across teams to deliver complex projects. Translate between specialists, build shared understanding and ownership, clear up technical and organisational dependencies, and state recommendations and specific requests plainly.
  • Investigate exploitation and national security threats in your area. Work with domain experts to understand how actors behave, spot coercion, exploitation or coordinated misuse, and turn case findings into detection criteria.
  • Link detection to action. With Policy and Operations, build review and escalation procedures, separate abusive activity from legitimate use, and strengthen interventions when new cases expose gaps.
  • Assess the whole defence system: remaining harm, detection quality and intervention results. Work with Data Science on better metrics and use expert feedback and operational outcomes to guide improvements.
  • Protect product and model launches by setting data, detection and review requirements, putting monitoring in place, investigating gaps and critical escalations, and improving defences based on post-launch findings.
  • Apply language models, agents and reusable tools to scale the work. Decide where automation or human judgement fits, and write documentation and processes that make results reliable and easy for others to extend.

Requirements

  • You think in systems, with enough breadth across technical, analytical, investigative and operational work to see how they connect, spot gaps and translate between specialists.
  • You have deep expertise in at least one of sextortion, trafficking, national security, jailbreaks or a closely related harm area. Backgrounds such as trust and safety investigations, counter-exploitation, threat intelligence, work on state-linked actors, or adversarial testing of AI safeguards are relevant. You do not need all of these areas.
  • You have investigated adversarial behaviour and turned partial, changing evidence into usable signals. You can link single cases to wider patterns while stating uncertainty and the risk of flagging legitimate activity.
  • You are proficient in SQL and Python and have written reliable, efficient production code for detection or data processing. Machine learning or software engineering experience is a plus.
  • You can use quantitative analysis and machine learning to measure risk and judge system effectiveness. You know the limits of your data and can reason about missed harms, false positives and the right intervention. Data science or measurement background is a plus.
  • You have set strategy and delivered complex, cross-functional technical projects. You can turn a vague problem into a plan, bring people with different priorities together, and see the work through to the end.
  • You write and speak clearly and concisely. You can explain technical findings, make concrete recommendations and show uncertainty and trade-offs to different audiences.
  • You can use language models and other automation to scale analytical, investigative or operational work, with evaluation and human oversight suited to the problem.

Conditions

The role is based in San Francisco and includes an on-call rotation that may require urgent work outside normal hours.

Some investigations involve sensitive content, including sexual, violent or otherwise disturbing material.

The company is an equal opportunity employer and does not discriminate on the basis of protected characteristics.

Background checks follow applicable law, and candidates with arrest or conviction records are considered in line with San Francisco, Los Angeles County and California fair chance rules. For unincorporated Los Angeles County roles, the company has stated that criminal history may have a direct, adverse and negative relationship with protecting entrusted hardware, returning hardware and data, and keeping non-public information confidential, which may lead to a conditional offer being withdrawn. The role also requires access to secure, protected systems and data.

Where you’d work

Part of the week in the office

The office

You can work from

  • United States

About the company

Company hidden

  • Industry: AI

Office in San Francisco, United States

Your chances

Still hiring, not crowded yet, and you'd be among the first.

  • 17 checks run
  • 7 good signs
  • 2 red flags

Still hiring?

11 checks

Actively hiring

In its favour4

  • Still on the company's own careers site, checked 1 h agoModerate evidence
  • Found in the last 48 hours, before the big job boardsModerate evidence
  • Specific about the basics: pay, place, level, stack and contract all statedSlight evidence
1 moreFewer
  • States its salarySlight evidence

Against it1

  • Listed again after an earlier listing of the same role closedModerate evidence

How crowded?

6 checks

Low

In its favour3

  • In its Early Window: not on the big job boards yetStrong evidence
  • Hybrid in San Francisco: only people nearby can take itSlight evidence
  • Senior level: far fewer people qualifySlight evidence

Against it1

  • Pays more than 97% of similar roles: that draws applicantsSlight evidence

Fits Me

How well does this role fit you?

Answer a few questions or drop your CV, and every role gets a fit score with the reasons, this one first.

  • Your field
  • Level
  • Stack
  • Work model
  • Salary floor
  • Must-haves
Details11 facts · Role, Location, Compensation, Employment, Company
Tech stack
  • Python
  • SQL
Type
Full-time
Industry
AI
Specialty
Backend
Region
United States
Pay period
Annual
Show 5 more factsShow less

Role

Category
Development
Specialty
Backend
Tech stack
  • Python
  • SQL

Location

Work model
Hybrid
Region
United States
Office
  • San Francisco, United States
Remote from
  • United States

Compensation

Salary
$220,000 - 425,000 / year
Pay period
Annual

Employment

Type
Full-time

Company

Industry
AI

Something wrong with this vacancy?

Similar vacancies

Share this vacancy

What's wrong with it?

The employer never sees who reported.

Reason