Principal Software Engineer

Description

The company builds Enterprise AI, and this role is the senior technical lead for it. You will co-own the architecture with the teams that share the platform and set the technical bar for a small, senior team that works remotely.

The product spans an AI gateway, a control plane, model serving, identity, metering and observability. It ships as both hosted and installed editions.

Responsibilities

• Own the architecture across the gateway, control plane, model serving, identity, metering and observability, for hosted and installed editions.
• Decide the gateway direction, including the data plane and policy-based routing that produces per-request metering and audit records.
• Set the direction for serving LLMs on Kubernetes, covering GPU scheduling and sharing, autoscaling, multi-tenant and disaggregated serving, and cost per token.
• Define how enterprises install, upgrade and operate the product, including Helm, air-gapped networks, preflight checks and supply-chain security.
• Run design reviews, decide whether to build or buy, and record each decision as an ADR.
• Write code on the critical path, and mentor senior engineers, including on AI-assisted development.
• Work with design partners and customers, and represent the company in the Kubernetes and ML infrastructure communities.

Requirements

• 10+ years in infrastructure or distributed systems, several of them as a technical lead.
• Experience shipping multi-service systems that enterprises run in production, including software they install on their own infrastructure.
• Deep Kubernetes skills: operators, CRDs, scheduling, Helm, and debugging at scale.
• Production experience with GPU workloads, LLM inference or AI gateways.
• Expert Go, strong Python, and solid CI/CD and infrastructure as code.
• You lead through writing and influence, use AI-assisted development tools fluently, and work well with autonomy on a distributed team.
• Inference performance work such as quantization, batching, KV-cache management, and disaggregated prefill and decode.
• AI gateway or LLM routing work such as failover, rate limiting, token metering and policy enforcement.
• Experience taking a new product to its first enterprise customers.

Conditions

The role is remote and part of a distributed team.

The company is a young, high-energy business that values openness, collaboration, risk-taking and continuous growth.

Equal-opportunity and legal notices apply, including the company's use of automated decision-making technology, with the right to opt out and appeal, and consent to process personal data for applications.

Benefits

• A competitive compensation package and a strong benefits plan.
• Work alongside experienced colleagues who help Fortune 500 and Global 2000 customers adopt next-generation cloud technologies.
• Contribute to open-source projects at the leading edge of the field.
• The company is ranked second only to AWS as a Leader for Container Management on G2.