Senior Software Engineer, Distributed Systems - NIM Factory

Description

The company builds the factory that turns AI models into NIMs (the company Inference Microservices), which are validated for performance and accuracy before release. The team designs and builds the infrastructure, pipelines, backends, Docker build, test harness, metrics, performance engineering and log ingestion behind this work.

The NIMs are meant to be easy to use and fast, and they must be tested across cloud, customer self-hosted and local the company GPU setups. You will help choose how models are optimized and served across heterogeneous clusters, with the goal of getting every AI model onto the company GPUs wherever they run.

Responsibilities

• Build a factory pipeline that takes an AI model as input and produces a deployable service validated across cloud, on-prem and Kubernetes environments.
• Work with the team to set technical strategy and roadmaps, and deliver quick iterations on the NIM factory.
• Design interfaces, data models and schemas, and extend observability across the pipeline and its compute infrastructure.
• Work with technical leaders to design and build scalable, reliable factory components.
• Collaborate with several AI model teams to learn their needs and build infrastructure that makes each team more productive.
• Define metrics and make improvements based on user feedback.
• Mentor colleagues and work across teams to help them and yourself grow.

Requirements

• Show a track record of building distributed systems, compute systems, backend services, microservices and cloud software using advanced programming skills.
• Have worked effectively with cross-functional teams, principal engineers and architects across organizational boundaries.
• Have mentored others and helped grow teams, and can change direction and expectations to meet customer needs.
• Have deep experience with containerized distributed applications using tools such as Docker, Kubernetes, Cloud Endpoints, Helm and Prometheus.
• Be enthusiastic about building microservice applications and automated build and test pipelines.
• Communicate well and lead efforts that span several functions.
• Have proven experience debugging and analyzing the performance of distributed microservices or cloud systems.
• Hold a BS or MS in Computer Science, Computer Engineering or a related field, or have equivalent experience.
• Have 8+ years of demonstrated experience developing performant microservices, cloud software or tooling.
• Nice to have: experience with event-driven applications using tools such as Temporal, Kafka, Redis or similar, and the ability to discuss the trade-offs between them.
• Nice to have: experience building and deploying containers for microservices, cloud and on-prem, including their CI/CD pipelines.
• Nice to have: prior work on large-scale full stack development.

Conditions

Applications are accepted until at least November 6, 2026.

This role fills an existing vacancy.

The company uses AI tools in parts of its recruiting process.

The company is an equal opportunity employer and does not discriminate on protected characteristics.