You and the process

With a person
Anna, in-house recruiter
9 years hiring engineers
30 minutes with a real recruiter
They read your CV with you, on a call, and say where the offers are being lost.
Didn’t find what you were looking for? Tell us what to build
doctolib

Senior IA Inference Engineer (x/f/m)

  • Hybrid
  • 6+ years
  • English (C1)

Salary

Not stated

Similar roles pay €75K - 105K a year · our estimate

AI summary

For members

The whole posting in a few lines. Sign up to read it here and on every role you open.

Sign Up to Read

Description

The company builds AI-driven healthcare products, including assistants for consultations, phone calls and patient health support.

This role sits in the ML Platform team (TIGER) within AI & Machine Learning. You will own the self-hosted LLM serving platform, which keeps the most sensitive patient data on infrastructure the company controls.

You will build and run the shared base that every AI feature team depends on: deploying, tuning and scaling open-weight models across the GPU fleet, so AI for care teams and patients becomes faster, safer and cheaper per unit across Europe.

The tech team builds products and features that make daily work easier for care teams and patients. The platform is one fully cloud-native system with web and mobile interfaces, adapted to country and specialty rules.

Responsibilities

  • Own production reliability and disaster recovery: set SLOs, load-test before launches, roll out and roll back safely, configure autoscaling, and take part in on-call and incident response.
  • Build the shared serving foundations with MLOps engineers: define the serving stack, plan GPU capacity, automate deployment of open-weight models (Gemma, Mistral, Llama, Qwen and fine-tuned variants) on Ray Serve / Anyscale, and help move toward Kubernetes-based orchestration.
  • Tune serving engines (vLLM, Triton Inference Server, TensorRT-LLM) to cut time-to-first-token, inter-token latency and cost per token, using batching, caching, quantization and parallelism across GPUs from L4 to H200.
  • Set up the model registry and promotion path: versioning, validation and promotion to production, benchmarking speed and output quality under realistic load, and maintaining the model catalog with the Agents Platform team.
  • Make performance and spend measurable: build Datadog dashboards, alerts and cost reports covering latency, GPU utilization and inference spend, and find resource efficiency gains.
  • Keep the stack secure and compliant: work with Security and Privacy on network isolation, data residency and auditability, and assess new models, engines such as SGLang, and GPU generations.

Requirements

  • If you do not match this profile exactly but think the role suits your skills, the team still encourages you to apply.
  • 5+ years of software or computer engineering experience, including hands-on production work with at least one LLM serving engine (vLLM, SGLang, TensorRT-LLM, TGI or Triton). Ideally you have chosen, introduced, tuned or scaled one, not only run it on defaults.
  • Strong Python and Terraform skills, and a working grasp of transformer inference: prefill vs. decode, KV cache, batching, and whether a model is compute- or memory-bound on a given GPU.
  • Solid experience running GPU workloads on Kubernetes or a managed Ray / Anyscale platform, with a performance mindset: reproducible benchmarks, reading profiler output, and decisions backed by numbers.
  • Comfortable owning production reliability, including observability, incident response and on-call.
  • Excellent communication and collaboration, high agency, a pragmatic build-vs-buy view, and comfort working in a scope that is still taking shape.
  • Fluent English.
  • Optional, posted as fantastic extras: experience with distributed serving frameworks (KServe, llm-d, NVIDIA Dynamo), including disaggregated prefill/decode or KV-cache-aware routing, or with serving Mixture-of-Experts, multimodal or many-LoRA-adapter setups on shared base models.
  • Optional, posted as fantastic extras: GPU kernel or deep profiling skills (CUDA, Triton, Nsight), or contributions to open-source inference projects.
  • Optional, posted as fantastic extras: experience in regulated sectors (healthcare, finance, public sector) and with GDPR, HDS, ISO 27001 or SOC 2.
  • Stack: Python, plus reading or debugging one systems language (C++, Rust or Go).
  • Stack: GCP, AWS or Azure. Serving on Ray Serve / Anyscale (GCP), with Kubernetes (GKE) and GPU scheduling as an emerging parallel target.
  • Stack: vLLM, Triton Inference Server and TensorRT-LLM in production, with SGLang and other new engines under evaluation.
  • Stack: self-hosted open-weight LLMs (Gemma, Mistral, Llama, Qwen), fine-tuned in-house. MLflow for tracking and registry, LiteLLM as gateway, Braintrust for evaluation and observability, Datadog for monitoring.
  • Stack: NVIDIA L4, A10G, L40S, RTX 6000 Pro, A100 (40GB/80GB) and H200 GPUs. Docker, Terraform, Helm and Kustomize, with GCP BigQuery / GCS or AWS S3 for storage.
  • Stack: any other tool you find relevant.

Benefits

  • Comprehensive basic-package health insurance at no cost for you and your children.
  • 25 days of paid vacation a year, plus up to 14 days of RTT.
  • Free mental health and coaching services through partner Moka.care.
  • Work from abroad for up to 10 days a year under the flexibility days policy.
  • Lunch vouchers (Swile card) worth €8.50 per working day, with €4.50 covered by the company.
  • A work council subsidy toward a sports club or creative class membership.
  • 50% reimbursement of your public transport subscription.
  • ParentCare Program: 100% of salary during the first month of birth leave and 75% during the second, covered by the company.
  • Enrollment in the long-term employee value sharing plan, DoctoGrowth.
  • For caregivers and workers with disabilities: an adjusted remote policy, extra days off for medical reasons, and psychological support.
  • Relocation support for international moves.
  • Access to top AI tools for coding and development, plus dedicated training.

Where you’d work

Part of the week in the office

Relocation offered

About the company

doctolib

  • Industry: HealthTech

Office in Paris, France

Also hiring in Berlin, Germany, Nantes, France and France

Your chances

Worth a look before you spend an evening tailoring a CV for it.

  • 20 checks run
  • 3 red flags

Still hiring?

13 checks

1 red flag

How crowded?

7 checks

2 red flags

Fits Me

How well does this role fit you?

Answer a few questions or drop your CV, and every role gets a fit score with the reasons, this one first.

  • Your field
  • Level
  • Stack
  • Work model
  • Salary floor
  • Must-haves
Details14 facts · Role, Location, Compensation, Employment, Company, Requirements
Tech stack
  • React
  • TypeScript
  • Python
  • Go
  • Java
  • Kotlin
  • Swift
  • AWS
  • Kubernetes
  • Docker
  • Terraform
  • C++
  • Rust
  • React Native
  • Azure
  • LLMs
  • CUDA
Seniority
Senior
Type
Full-time
Industry
HealthTech
Specialty
Backend
Region
Europe
Show 8 more factsShow less

Role

Category
Development
Specialty
Backend
Seniority
Senior
Experience
5+ years
Tech stack
  • React
  • TypeScript
  • Python
  • Go
  • Java
  • Kotlin
  • Swift
  • AWS
  • Kubernetes
  • Docker
  • Terraform
  • C++
  • Rust
  • React Native
  • Azure
  • LLMs
  • CUDA

Location

Work model
Hybrid
Region
Europe
Office
  • Paris, France
Relocation
Offered

Compensation

Salary
Salary by agreement
Pay period
Annual

Employment

Type
Full-time

Company

Industry
HealthTech

Requirements

Languages
  • English (C1)

Something wrong with this vacancy?

Similar vacancies

Share this vacancy

What's wrong with it?

The employer never sees who reported.

Reason