Principal Software Engineer, LLM Gateway

Description

The team builds the AI Trust Layer, the gateway that every the company product and every customer-built agent uses to reach models from OpenAI, Anthropic, Google and other commercial and open-weight providers.

The role sets technical direction for a secure, globally distributed, high-throughput proxy that handles text, image, voice and newer modalities, with intelligent routing, rate limiting and cost controls.

The proxy must run the same way in the public cloud, in government and sovereign clouds, and in customer-hosted deployments.

You will own the LLM infrastructure behind it, sourcing and balancing model capacity across vendors, tiers and regions, with data residency guaranteed in every geography.

You will also shape how customers see and control AI in their organizations: which models their users and automations can access, including models they bring themselves, under what conditions, and with what policies, data protection and audit trail.

You will raise the engineering bar for the team, work closely with product, security, legal and model vendors, and help define how the team builds software with AI agents.

The company is a growth company that moves fast and is focused on enterprise automation software.

Responsibilities

• Lead the architecture and technical direction of complex backend and cloud-native services across the platform.
• Design scalable, secure, reliable and high-performing distributed systems that run in cloud, on-premises and hybrid environments.
• Work with engineering leaders, architects, product managers and other teams to shape technical solutions for high-impact product and platform areas.
• Drive technical planning and roadmap input for complex services, platform capabilities and cross-team initiatives.
• Find long-term problems in scalability, reliability, security, performance and cost, and lead the design of lasting fixes.
• Set high standards for system design, code quality, testing, observability, automation, incident management and engineering practice.
• Mentor engineers through design reviews, technical discussions, architecture decisions and hands-on feedback.

Requirements

• 7+ years of software engineering experience, including strong backend and cloud development.
• Hands-on experience with one or more modern languages such as C, Java, Python or Go.
• Strong experience designing and building distributed systems, microservices, APIs and cloud-native architectures.
• A proven record of influencing technical direction and driving architecture decisions across teams.
• Experience building and running production-grade services in Azure, AWS or GCP.
• Good working knowledge of containers, Kubernetes, CI/CD, monitoring, logging, observability and production operations.
• Strong grasp of system design, data structures, algorithms, multithreading, asynchronous programming and performance optimization.
• Experience with relational and/or NoSQL databases, messaging systems, event-driven architectures or high-scale data processing.
• Ability to weigh customer experience, reliability, security, scalability and cloud cost when making technical decisions.
• Strong technical leadership, ownership and communication skills, with the ability to guide teams through ambiguity.
• Interest in automation, AI, developer platforms, enterprise software or large-scale cloud systems is a plus.

Conditions

Work location and time can be flexible. The mix of hybrid, office-based and remote workers varies by team and by business need.

Applications are reviewed on a rolling basis with no fixed deadline. The window may close early if a qualified candidate is selected or may change based on application volume.

Candidates who do not meet every point above are still encouraged to apply.