AI Engineer

Description

The company, founded in 2004, makes no-code network automation software used by IT operations teams to run hybrid multi-cloud networks. Its platform automates diagnostic troubleshooting, outage prevention and protected change management. More than 2,500 large enterprises and managed service providers use it.

The team is hiring a senior AI engineer to design and build production agent and retrieval-augmented generation (RAG) systems that make platform automation smarter and more dependable. The work mixes hands-on building with system-level decisions covering architecture, evaluation, scale, observability and reliability once systems are live. You will be comfortable with unclear requirements and will move quickly from prototype to production, with a strong focus on quality, safety and real impact.

Responsibilities

• Design and build the core of an enterprise agent platform: orchestration patterns such as ReAct, Plan-and-Execute and Supervisor, plus tool execution, context and memory handling, and safety guardrails.
• Design governance and execution controls for agents, including human-in-the-loop approval flows, per-tenant permission isolation, policy enforcement and secure execution.
• Build reusable agent skills and standard tool interfaces, and grow a scalable tool ecosystem tied closely to platform features and business workflows.
• Design and run post-training for LLMs, including domain-specific supervised fine-tuning, DPO or RLHF preference alignment, and parameter-efficient methods such as LoRA, to keep improving performance on network operations tasks.
• Build a self-learning feedback loop that turns production traces, user feedback and evaluation results into high-quality datasets for improving prompts, skills, models and retrieval.
• Analyse and improve LLM behaviour in instruction following, tool calling, structured output, context understanding, reasoning stability and hallucination reduction.
• Build production LLM and agent evaluation frameworks and automated regression pipelines, covering benchmark datasets, deterministic checks, LLM-as-a-judge scoring, tool-call validation, retrieval quality and end-to-end task success.
• Set release quality gates and hallucination detection for AI features so accuracy, reliability or performance regressions do not reach production.
• Build AI observability with distributed tracing, structured logs, metrics, dashboards and alerts.
• Quickly diagnose and fix production AI failures, such as hallucinations, wrong tool choices, invalid tool parameters, agent loops, retrieval degradation, structured-output errors, latency regressions and unexpected model changes.
• Build reliable backend services for AI and agent workloads, using async and concurrent processing, retries, timeouts, caching, rate limiting and fault isolation.
• Keep reducing latency, throughput, token use and infrastructure cost to meet platform SLAs at large production scale.
• Independently find and fix complex issues across prompts, models, RAG, tools, agent workflows, backend services and infrastructure.
• Lead technical design for critical modules and system-level features, making sure solutions fit the platform architecture, security requirements, engineering standards and product needs.
• Use production data, evaluation results and benchmarks to drive technical improvements, and work closely with engineering, product, QA and other teams to ship to production.
• Prototype, benchmark and productionise emerging approaches such as GraphRAG, knowledge graphs, MCP, LLM post-training and agent self-learning, to improve grounding, multi-hop reasoning and domain expertise.
• Evaluate agent frameworks and supporting infrastructure, including LangChain, LangGraph, AutoGen and LlamaIndex, and recommend how platform architecture and product technology should evolve.
• Track developments in LLM and agent technology and turn promising ones into measurable, testable, production-ready capabilities.

Requirements

• A bachelor's degree or higher in computer science, artificial intelligence, electrical engineering or a related technical field. A master's or PhD is preferred; equivalent practical experience will also be considered.
• At least 3 years in software engineering, machine learning or applied AI, including at least 2 years building, deploying and running production LLM or agent applications. You must have delivered at least one LLM-powered feature end to end and owned its operation and improvement after launch.
• A deep understanding of agent architectures and LLM behaviour, including instruction following, tool-calling behaviour and context sensitivity. You have hands-on experience building multi-step workflows that involve reasoning, tool execution, state management, structured outputs, validation and error recovery.
• A proven record of diagnosing and fixing production LLM and agent failures, including hallucinations, incorrect tool calls, retrieval degradation, agent loops, structured-output errors, latency regressions and regressions caused by prompt or model changes.
• Strong Python and distributed backend skills, including API and service development, async and concurrent programming, retries, timeouts, caching, rate limiting, testing, logging and cross-service performance debugging.
• Hands-on experience designing evaluation systems for LLM applications, including dataset construction, metric definition, regression testing and release quality gates.
• A solid understanding of security risks in LLM and agent applications, including prompt injection, data leakage, unsafe tool execution, permission boundaries and uncontrolled agent autonomy.
• You can design, build, debug, deploy and run complex production systems on your own.
• Preferred: strong experience with RAG and advanced retrieval, including embeddings, vector and hybrid search, reranking, chunking, grounding and citation, and multi-hop retrieval, knowledge graphs or GraphRAG.
• Preferred: familiarity with agent frameworks such as LangGraph, LangChain, AutoGen and LlamaIndex, and with standard protocols such as MCP (Model Context Protocol) for connecting agents to external tools and systems. Understanding the underlying architecture matters more than knowing any one framework.
• Preferred: experience designing agent runtime mechanisms, including human-in-the-loop workflows such as risk classification, approval, interruption and pause/resume, and context management for long-running agents, including state persistence, history compression and memory systems.
• Preferred: experience with LangSmith or similar LLM observability and evaluation platforms.
• Preferred: experience with LLM fine-tuning, including LoRA or other parameter-efficient techniques.
• Preferred: experience applying LLM technology to networking, infrastructure, cybersecurity, observability or other complex technical domains.
• Preferred: fluency in both English and Chinese, with strong communication and collaboration across regions.

Conditions

Base pay is estimated at $150,000 to $180,000, plus a bonus. The actual salary depends on factors including the market and individual qualifications, assessed objectively during interviews. The range is a guide and may change.

The company conducts holistic individual background reviews in line with applicable law.

Benefits

• Total compensation is designed to recognise the impact people have on company goals, beyond base pay.
• Benefits include 401(k) and medical and dental coverage, with more detail available from the recruiter on the total rewards approach.