You and the process

With a person
Anna, in-house recruiter
9 years hiring engineers
30 minutes with a real recruiter
They read your CV with you, on a call, and say where the offers are being lost.
Didn’t find what you were looking for? Tell us what to build
PolarGrid

Staff Software Engineer, Inference Routing

  • Remote
  • 6+ years

Salary

$220,000 - 265,000/ year

AI summary

For members

The whole posting in a few lines. Sign up to read it here and on every role you open.

Sign Up to Read

Description

The company is an early-stage Canadian business that runs managed AI inference. Customers bring their own models, and the company serves them as fast, reliable production endpoints on GPU servers across many sites.

The team is about five engineers.

Responsibilities

  • You will own the routing layer, the service that picks which GPU handles each request.
  • You will build the router, run it in production and set its technical direction.
  • Most of your time will go to designing and writing the router and the placement logic behind it.
  • You will work closely with the engineers who run the network and the GPU sites.
  • You will help hire the engineers who join the team.
  • Each request must reach a GPU that is healthy, has spare room and is close enough to respond quickly.
  • Capacity and health shift constantly across many sites, so you will make fast routing decisions from information that is always a little stale.
  • When a site or server fails, traffic must move elsewhere without customers noticing.

Requirements

  • You have deep knowledge of how networks, proxies and load balancers work, gained from building or running production systems.
  • You have strong distributed systems fundamentals, including consistency, failure handling, retries and backpressure.
  • You write production code in a systems language such as Go or Rust.
  • You measure latency at the tail, not only the average, and can trace where it comes from.
  • You have been on call for systems you built.
  • You have set technical direction that other engineers built on.
  • You write clearly.
  • You use AI tools effectively and verify their output.
  • GPU or machine learning experience is not required. What matters is that you understand the underlying systems well and want to build a new one from scratch.

Conditions

The role is remote. You must be based in Canada and able to work from there.

Where you’d work

Fully remote

You can work from

  • Canada

About the company

PolarGrid

  • Industry: AI

Your chances

Worth a look before you spend an evening tailoring a CV for it.

  • 20 checks run
  • 1 red flag

Still hiring?

13 checks

No red flags

How crowded?

7 checks

1 red flag

Fits Me

How well does this role fit you?

Answer a few questions or drop your CV, and every role gets a fit score with the reasons, this one first.

  • Your field
  • Level
  • Stack
  • Work model
  • Salary floor
  • Must-haves
Details11 facts · Role, Location, Compensation, Company
Tech stack
  • Go
  • Rust
Seniority
Staff
Industry
AI
Specialty
Backend
Region
Canada
Pay period
Annual
Show 5 more factsShow less

Role

Category
Development
Specialty
Backend
Seniority
Staff
Experience
6+ years
Tech stack
  • Go
  • Rust

Location

Work model
Remote
Region
Canada
Remote from
  • Canada

Compensation

Salary
$220,000 - 265,000 / year
Pay period
Annual

Company

Industry
AI

Something wrong with this vacancy?

Similar vacancies

Share this vacancy

What's wrong with it?

The employer never sees who reported.

Reason