Forward Deployed Engineer, Inference

Description

The company builds foundation AI models and end-to-end products for enterprises, with a security-first approach. Its team of researchers, engineers and designers trains frontier models and deploys them for businesses building AI systems.

The team makes these large language models run reliably, securely and at high performance inside customers' own infrastructure and data boundaries. This role sits between the core engineering teams and the company's strategic partners, and focuses on hard, loosely defined inference problems specific to each customer.

The company is headquartered in Toronto and has offices in London, New York City, San Francisco, Montreal, Paris, Berlin and Seoul.

Responsibilities

• Work inside customer engineering teams to connect the models to their live, complex production systems.
• Design model serving setups, including multi-tenant architectures, smart model routing and failover across datacenters for multi-cluster environments.
• Tune AI-in-a-box and private deployments, weighing Kubernetes multi-cluster scale limits against model performance.
• Benchmark and profile customer LLM workloads, and find the causes of slowdowns such as slow generation using metrics, dashboards and distributed logs.
• Help clients run and support the models on new accelerators, including GPUs, TPUs, AMD hardware and custom ASICs.
• Turn problems found in the field into platform improvements and direct feedback to the core product.
• Help customers set up sound deployment practices, SLOs and observability from day one.

Requirements

• At least 8 years of engineering experience running production infrastructure, building large distributed systems or deploying enterprise software in complex environments.
• Deep hands-on skill with production Kubernetes across multi-cluster and hybrid or multi-cloud setups on GCP, Azure, AWS or OCI.
• A solid understanding of LLM architecture and inference, including how attention, compute-bound prefill and memory-bound decode affect live performance.
• Practical experience with inference engines and optimization stacks such as vLLM, TensorRT-LLM, lmd or Dynamo.
• Familiarity with distributed serving methods, such as prefill/decode disaggregated serving, distributed KV caching and multi-GPU workloads.
• A proven record of leading technical work with customers, scoping vague problems and guiding senior technical stakeholders.
• High agency and persistence, with the ability to write production code the same day while handling client-side deployment politics or blockers.
• Strong programming skills in Python, Go or C++, and comfort moving between backend data pipelines and infrastructure tuning.
• Candidates who meet only part of this list are still encouraged to apply.

Conditions

Full-time position.

Remote-friendly, with the option to work from any of the company's offices.

The company will use AI-enabled tools to help screen and assess applicants against the role's criteria, but recruiters can still review any application.

The company provides equal opportunities and welcomes applicants from all backgrounds.

Benefits

• A weekly lunch stipend of $75 or £75, or the equivalent in your local currency.
• Full health and dental coverage, plus a separate mental health budget.
• RRSP matching, a 401K and a pension scheme.
• 100% parental leave top-up for up to 6 months, for either parent.
• Annual enrichment budget for arts and culture, fitness and wellness, quality time, and a workspace improvement credit.
• An education and learning stipend for conferences, courses and coaching.
• 6 weeks of paid vacation (30 working days).
• A budget to travel to other offices if you work remotely, plus an annual company offsite.
• A $500 home office stipend.
• For those in an office: a daily lunch program, snacks and regular community and social events.
• For those far from an office: a co-working benefit to work alongside others in your city.