You and the process

With a person
Anna, in-house recruiter
9 years hiring engineers
30 minutes with a real recruiter
They read your CV with you, on a call, and say where the offers are being lost.
Didn’t find what you were looking for? Tell us what to build
Perplexity

Member of Technical Staff (Web Access Engineer, Search Crawler)

  • Hybrid
  • 6+ years

Salary

Not stated

Similar roles pay £90K - 130K a year · our estimate

AI summary

For members

The whole posting in a few lines. Sign up to read it here and on every role you open.

Sign Up to Read

Description

Perplexity is hiring a Web Access Engineer for the Crawler team to build reliable fetching and rendering systems for AI-powered search. You'll work on freely available public web content, within site permissions and approved access policies, and develop the infrastructure that supplies content to downstream indexing and retrieval systems.

Your focus will be diagnosing network errors, page-loading issues, and incomplete rendering. You'll trace problems from network protocols through client-side execution, distinguish technical failures from intentional access restrictions, and turn findings into production fixes that improve content quality and efficiency at scale.

Responsibilities

  • Own fetching and rendering improvements end to end, from investigation through implementation and production rollout, partnering with crawler engineers and SREs.
  • Investigate fetch and rendering failures across DNS, TLS, HTTP, caching, redirects, and client-side rendering, and classify root causes.
  • Improve page loading, completion detection, and content capture to preserve relevant content and handle dynamic pages reliably.
  • Ensure the crawler respects site-imposed access restrictions and rate limits, identifies itself clearly, and retries only transient technical errors.
  • Build tools to reproduce failures, compare crawler output against expected page content, and turn findings into regression tests.
  • Define metrics for fetching reliability, content completeness, and error classification, and use them to prioritize improvements.

Requirements

  • 5+ years of software engineering experience in web infrastructure, browsers, CDNs, or crawling, with strong programming skills in at least one of Go, Python, Rust, or C++.
  • Deep understanding of HTTP/1.1, HTTP/2, and HTTP/3, including status semantics, caching, conditional requests, content encoding, and redirects; strong knowledge of TLS, DNS, cookies, and sessions.
  • Understanding of CDN caching and content delivery, HTTP error handling, retry backoff, and server-defined rate limits.
  • Knowledge of page lifecycle, network idle, service workers, client-side rendering, and shadow DOM; experience debugging JavaScript and fluency with Chrome DevTools, CDP, HAR files, curl, and packet captures.
  • Comfort working independently when the path forward is not fully defined.
  • Experience developing CDN, caching, or web performance systems.
  • Experience operating crawlers, headless browsers, or distributed content-acquisition systems at scale.
  • Contributions to a browser engine or JavaScript runtime.

Where you’d work

Part of the week in the office

You can work from

  • United Kingdom

About the company

Perplexity

  • Industry: AI

Offices in London, United Kingdom, Dublin, Ireland

Also hiring in San Francisco, United States, New York City, United States, Palo Alto, United States and 4 more places

3 of their 36 open roles are remote

Your chances

Worth a look before you spend an evening tailoring a CV for it.

  • 19 checks run
  • 1 red flag

Still hiring?

13 checks

No red flags

How crowded?

6 checks

1 red flag

Fits Me

How well does this role fit you?

Answer a few questions or drop your CV, and every role gets a fit score with the reasons, this one first.

  • Your field
  • Level
  • Stack
  • Work model
  • Salary floor
  • Must-haves
Details13 facts · Role, Location, Compensation, Employment, Company
Tech stack
  • Python
  • Go
  • JavaScript
  • C++
  • Rust
Seniority
Staff
Type
Full-time
Industry
AI
Specialty
Backend
Region
United Kingdom
Show 7 more factsShow less

Role

Category
Development
Specialty
Backend
Seniority
Staff
Experience
5+ years
Tech stack
  • Python
  • Go
  • JavaScript
  • C++
  • Rust

Location

Work model
Hybrid
Region
United Kingdom
Offices
  • London, United Kingdom
  • Dublin, Ireland
Remote from
  • United Kingdom

Compensation

Salary
Salary by agreement
Pay period
Annual

Employment

Type
Full-time

Company

Industry
AI

Something wrong with this vacancy?

Similar vacancies

Share this vacancy

What's wrong with it?

The employer never sees who reported.

Reason