You and the process

With a person
Anna, in-house recruiter
9 years hiring engineers
30 minutes with a real recruiter
They read your CV with you, on a call, and say where the offers are being lost.
Didn’t find what you were looking for? Tell us what to build
Equinix

Production Reliability Engineer, AI Platform

  • Hybrid
  • 3-6 years

Salary

$138,000 - 208,000/ year

AI summary

For members

The whole posting in a few lines. Sign up to read it here and on every role you open.

Sign Up to Read

Description

The company provides digital infrastructure worldwide, and this is a new position on the team.

You will keep the company's GenAI products and platforms stable and reliable, and make them more resilient over time.

You will write code and automation that prevents failures instead of only reacting to them: defining SLOs, building monitoring and alerting, hardening services against dependency and capacity failures, and testing failure and recovery before they reach production.

You will lead incident response and turn every incident into a lasting fix.

You will work with senior engineers and with product, platform and security partners to keep AI products, agents, connectors and self-hosted models robust, secure, observable and fast.

Success means GenAI products meet their SLOs, releases ship without avoidable incidents, alerts catch failures before users report them, each incident ends with a documented fix, the same incident does not recur, load and recovery tests run on schedule, launches scale without capacity problems, manual operational work falls each quarter, and you take on larger, less defined work with less guidance over time.

Responsibilities

  • Build code and automation that make GenAI products and platforms more resilient, including retries, timeouts, circuit breakers, rate limiting, graceful degradation, and fallback between models and providers
  • Run load tests, failure-injection tests and game days to find weak points before users do, and fix what they reveal
  • Harden Kubernetes services with health checks, autoscaling, disruption budgets, resource limits and multi-zone deployment
  • Plan capacity ahead of launches and growth, covering model-provider quotas, GPU and inference capacity, and database headroom
  • Maintain backup, restore and disaster-recovery plans, and test recovery on a regular schedule
  • Automate repeated operational tasks and reduce manual work every quarter
  • Define and track SLIs and SLOs with product owners for availability, latency, error rate, token cost and quality scores
  • Instrument services with logs, metrics and traces (e.g., OpenTelemetry), from the user request through agent, tool and model calls
  • Build and tune dashboards and alerts for user-facing failures, and remove alerts that are noisy or not actionable
  • Check release readiness for monitoring, alerts, runbooks, capacity, security review and rollback plans
  • Deploy through CI/CD and GitOps pipelines using containers and infrastructure as code, with canary rollouts and fast rollback
  • Join the on-call rotation, respond to incidents, and write blameless post-incident reviews
  • Track post-incident actions to completion so the same failure does not happen twice
  • Write and update runbooks, and turn recurring incidents into automation or fixes
  • Support the reliability of self-hosted open-weight and small language models, covering serving, GPU usage, scaling, rollout and rollback
  • Monitor model and provider latency, errors, rate limits and cost, and flag capacity and cost risks
  • Apply security, patching, secrets and data-boundary requirements to every deployment

Requirements

  • 1-4 years in site reliability, DevOps, platform or software engineering, with hands-on work on production systems
  • Bachelor's degree in computer science, engineering or a related field, or equivalent experience
  • Proficiency in Python or Go, with solid habits in version control, testing and code review
  • Experience with Linux, containers, Kubernetes and a major cloud platform (Google Cloud preferred)
  • Experience with CI/CD and infrastructure as code (e.g., Terraform)
  • Working knowledge of monitoring, logging, tracing and alerting
  • Understanding of how distributed systems fail and of common resilience patterns
  • Experience with on-call or incident response, and willingness to join an on-call rotation
  • Familiarity with LLM applications, including model APIs, retrieval, agents and tool calls
  • Clear written and verbal communication
  • Experience running GenAI or ML systems in production, including managed model platforms (e.g., Vertex AI) or model serving (e.g., vLLM)
  • Experience with observability tools (e.g., OpenTelemetry, Prometheus, Grafana) and incident tools (e.g., PagerDuty)
  • Experience with GitOps (e.g., Argo CD), Helm and canary deployments
  • Experience with load testing or chaos engineering
  • Experience with secrets management (e.g., Vault) and security scanning
  • Experience building automation or AI agents that reduce operational work
  • Understanding of how GenAI products behave in production, including model latency, rate limits, quotas, streaming and token cost

Conditions

Target base pay in Canada (Toronto Office, TRO) is 102,000 to 152,000 CAD per year

Target base pay in the United States (Dallas Infomart Office, DAI) is 138,000 to 208,000 USD per year

Pay ranges are set by role, level and location, may change over time, and individual pay depends on skills, experience, and education or training

The ranges cover base pay only, not bonus, equity or benefits; employees may be eligible for a bonus, and equity may be offered depending on the position

The company uses artificial intelligence in its hiring process

The company is committed to an open hiring process for all individuals, including people with disabilities, and will provide accommodation on request

Benefits

  • US: health, life, disability and voluntary insurance for you and eligible family members
  • US: a retirement plan that you and the company may contribute to
  • US: accrued paid time off each pay period and paid holidays, subject to eligibility and plan terms
  • Canada: healthcare coverage that complements the provincial system, plus life, disability and optional benefit plans for you and eligible family members
  • Canada: Defined Contribution Pension Plan (DCPP), Group Retirement Savings Plan (RRSP) and Tax-Free Savings Plan (TSFA), which you may enroll in
  • Canada: vacation, personal time and paid holidays, subject to eligibility and plan terms, which may change at the company's discretion
  • Employee Assistance Program available to all employees

Where you’d work

Part of the week in the office

About the company

Equinix

Offices in Toronto, Canada, Dallas, United States, Canada

Also hiring in London, United Kingdom, United Kingdom, United States and 14 more places

Your chances

Worth a look before you spend an evening tailoring a CV for it.

  • 19 checks run
  • 1 red flag

Still hiring?

13 checks

1 red flag

How crowded?

6 checks

No red flags

Fits Me

How well does this role fit you?

Answer a few questions or drop your CV, and every role gets a fit score with the reasons, this one first.

  • Your field
  • Level
  • Stack
  • Work model
  • Salary floor
  • Must-haves
Details12 facts · Role, Location, Compensation, Employment
Tech stack
  • Python
  • Go
  • Google Cloud
  • Kubernetes
  • Terraform
  • Linux
  • CI/CD
  • Argo CD
  • Helm
  • Prometheus
  • Grafana
Type
Full-time
Equity
Equity offered
Specialty
SRE
Region
Canada
Pay period
Annual
Show 6 more factsShow less

Role

Category
DevOps & Infrastructure
Specialty
SRE
Experience
4+ years
Tech stack
  • Python
  • Go
  • Google Cloud
  • Kubernetes
  • Terraform
  • Linux
  • CI/CD
  • Argo CD
  • Helm
  • Prometheus
  • Grafana

Location

Work model
Hybrid
Region
Canada
Offices
  • Toronto, Canada
  • Dallas, United States
  • Canada
Remote from
  • Canada

Compensation

Salary
$138,000 - 208,000 / year
Pay period
Annual
Equity
Equity offered

Employment

Type
Full-time

Something wrong with this vacancy?

Similar vacancies

Share this vacancy

What's wrong with it?

The employer never sees who reported.

Reason