AI Infrastructure Engineer [Tooling]

Description

The company is backed by Foundation Capital and builds multi-agent AI systems that automate complex business workflows on platforms such as SAP, Salesforce, Workday, Snowflake and Databricks.

The AI Platform team builds and runs the infrastructure that keeps these agents reliable, secure and consistent across customer environments. That includes the cloud base on Kubernetes, networking and automation, plus the AI layer: model serving, agent runtimes, evaluation and release tooling, and internal CLIs and services.

This work lets the company bring enterprise customers on board under strict security and compliance rules without slowing developers, reduces integration risk, standardizes operations across hyperscalers, and lets product teams ship AI features safely.

Responsibilities

• Build, ship and own the tooling that moves AI agents from a developer's machine into production in customer clouds.
• Write software in Python or Go, including CLIs, Kubernetes controllers and internal services that run alongside LLM serving stacks such as vLLM.
• Deploy and tune LLM inference, covering GPU scheduling, autoscaling, batching, and the balance between cost and latency.
• Create infrastructure for multi-agent systems: agent runtimes, tool and connector execution, sandboxing, state handling and safe rollouts of new agent versions.
• Run MLOps workflows: versioning for models and prompts, evaluation pipelines, artifact registries and promotion from staging to production.
• Write Kubernetes operators and CRD-based controllers that define how AI workloads are deployed and managed.
• Design and run production AI agent infrastructure using Kubernetes and Terraform.
• Maintain the portable infrastructure across Azure, Oracle Cloud, GCP and AWS, and take primary ownership of at least one of these clouds.
• Improve observability by tracing agent and LLM calls and tracking token and GPU use, latency, errors and cost.
• Harden security and reliability for workloads in customer environments that face strict compliance requirements.
• Design and own CI/CD for infrastructure, services and model or agent releases, using GitOps and pipeline-as-code.
• Build self-service tools for environment provisioning, secret rotation, dependency updates, compliance checks and release management.
• Work closely with AI, product and platform engineers to keep delivery moving.
• Automate repeated work. Anything automated should be observable and maintained like production code.

Requirements

• 5+ years in infrastructure, platform or backend engineering, or 7+ years for senior roles.
• Strong Python or Go skills, with a record of shipping tools, CLIs or services that other engineers depend on.
• Hands-on production experience serving LLMs with vLLM or a comparable stack such as TGI, Triton, SGLang or Ray Serve.
• Experience building infrastructure for AI agents or LLM applications, including orchestration, tool calling, evaluation or agent runtimes.
• Practical MLOps experience: model and prompt versioning, evaluation and release pipelines, and observability for ML systems.
• Strong Kubernetes skills, including writing or extending operators and controllers with Kubebuilder, Operator SDK, controller-runtime or Kopf.
• Strong Terraform skills and hands-on work with at least one of Azure, Oracle Cloud, GCP or AWS.
• A proven track record designing and owning CI/CD systems, GitOps deployments and release automation.
• Solid networking fundamentals.
• A proven record of deploying and operating real production systems.
• Nice-to-Have: GPU infrastructure experience with NVIDIA GPU Operator, MIG, KServe, or Karpenter-style autoscaling for inference.
• Nice-to-Have: Familiarity with agent frameworks and protocols such as LangGraph, Model Context Protocol or the OpenAI Agents SDK.
• Nice-to-Have: Use of LLM observability tools such as OpenTelemetry, Langfuse or Arize.
• Nice-to-Have: Experience with FluxCD, ArgoCD or other GitOps controllers.
• Nice-to-Have: Cloud security and DevSecOps work, including security gates, SBOM generation and compliance checks in CI/CD.
• Nice-to-Have: Deep networking knowledge of service mesh, private connectivity and multi-cloud networking.
• Nice-to-Have: Open-source contributions to AI infrastructure or Kubernetes ecosystem projects.
• Nice-to-Have: Enjoys clean, unified infrastructure and has a builder mindset.
• Nice-to-Have: Writes production-quality code, not just configuration.
• Nice-to-Have: Keeps up with the fast-moving AI infrastructure space and learns new tools readily.
• Nice-to-Have: Can own and defend infrastructure decisions, with clear reasoning about trade-offs.
• Nice-to-Have: Experience in startups, big tech or both.
• Nice-to-Have: Self-driven, with a strong sense of ownership.
• Nice-to-Have: Enjoys building foundational systems that other engineers rely on.

Conditions

This is a hands-on individual contributor role open at mid to senior level.

Mid-level engineers own well-scoped components end to end. Senior engineers also set technical direction, make architectural decisions across the platform and mentor others.

The level is set based on experience and interviews.

Benefits

• Work on cutting-edge AI infrastructure with real impact for enterprise customers.
• Own core systems at an early-stage company that is growing fast.
• Work with a sharp, execution-focused founding team.
• Help shape how AI agents are deployed across some of the largest enterprise platforms in the world.