AI Engineer

Description

The company is a no-code network automation vendor founded in 2004. Its Next-Gen platform helps IT operations teams scale hybrid multi-cloud connected networks by automating diagnostic troubleshooting, outage prevention and protected change management. Over 2,500 large enterprises and managed services providers use it.

The company is seeking a Senior AI Engineer to design and build production-grade agent and RAG systems that drive intelligent, reliable automation on the platform. The role mixes hands-on coding with system-level work, covering architecture, evaluation, scalability, observability and reliability once systems are live. The ideal person handles ambiguity well, moves fast from prototype to production, and cares about quality, safety and real-world impact.

Responsibilities

• Design and implement core capabilities for an enterprise Agent platform: orchestration patterns such as ReAct, Plan-and-Execute and Supervisor, tool execution, context and memory management, and safety guardrails.
• Build enterprise Agent execution and governance mechanisms, including human-in-the-loop approval, multi-tenant permission isolation, policy enforcement and secure execution controls.
• Build reusable Agent Skills and standardized tool interfaces, and grow a scalable tool ecosystem tightly integrated with platform features and business workflows.
• Design and run LLM post-training work, including domain-specific Supervised Fine-Tuning (SFT), DPO/RLHF preference alignment and parameter-efficient methods such as LoRA, to keep improving performance in network operations.
• Build an Agent self-learning loop that turns production traces, user feedback and evaluation results into high-quality datasets for improving prompts, skills, models and retrieval.
• Analyze and improve LLM behavior in instruction following, tool calling, structured output, contextual understanding, reasoning stability and hallucination reduction.
• Build production-grade LLM and Agent evaluation frameworks and automated regression pipelines, covering benchmark datasets, deterministic checks, LLM-as-a-Judge, tool-call validation, retrieval-quality checks and end-to-end task success metrics.
• Set release quality gates and hallucination detection for AI features so that major accuracy, reliability or performance regressions do not reach production.
• Build AI system observability, including distributed tracing, structured logging, metrics, dashboards and alerting.
• Quickly diagnose and fix production AI failures such as hallucinations, wrong tool selection, invalid tool parameters, Agent loops, retrieval degradation, structured-output errors, latency regressions and unexpected model changes.
• Design and build highly reliable backend services for AI and Agent workloads, using async and concurrent processing, retries, timeouts, caching, rate limiting and fault isolation.
• Keep optimizing latency, throughput, token use and infrastructure cost to meet platform SLAs at large production scale.
• Independently diagnose and fix complex AI issues across prompts, models, RAG, tools, Agent workflows, backend services and infrastructure.
• Lead technical design for critical modules and system-level capabilities, making sure solutions fit the platform architecture, security requirements, engineering standards and product needs.
• Use production data, evaluation results and benchmarks to drive technical improvements, and work closely with Engineering, Product, QA and other teams to ship to production.
• Prototype, benchmark and productionize emerging techniques such as GraphRAG, Knowledge Graphs, MCP, LLM post-training and Agent self-learning to improve grounding, multi-hop reasoning and domain depth.
• Assess Agent frameworks and supporting infrastructure, including LangChain, LangGraph, AutoGen and LlamaIndex, and recommend changes to platform architecture and product technology strategy.
• Track developments in LLM and Agent technology and turn promising ones into measurable, testable, production-ready engineering work.

Requirements

• A bachelor's degree or higher in Computer Science, Artificial Intelligence, Electrical Engineering or a related technical field. A master's or PhD is preferred; equivalent practical experience is also considered.
• At least 3+ years in software engineering, machine learning or applied AI, including 2+ years building, deploying and operating production LLM or Agent applications. You must have delivered at least one LLM-powered feature end to end and owned its ongoing operation and improvement after launch.
• Deep knowledge of Agent architectures and LLM behavior, including instruction following, tool-calling behavior and context sensitivity. Hands-on experience building multi-step workflows with reasoning, tool execution, state management, structured outputs, validation and error recovery.
• Proven ability to diagnose and fix production LLM and Agent failures, including hallucinations, incorrect tool calls, retrieval degradation, Agent loops, structured-output errors, latency regressions and regressions caused by prompt or model changes.
• Strong Python and distributed backend skills, including API and service development, async and concurrent programming, retries, timeouts, caching, rate limiting, testing, logging and cross-service performance debugging.
• Hands-on experience designing evaluation systems for LLM applications, including dataset construction, metric definition, regression testing and release quality gates.
• Strong understanding of security risks in LLM and Agent applications, including prompt injection, data leakage, unsafe tool execution, permission boundaries and uncontrolled Agent autonomy.
• Ability to design, build, debug, deploy and run complex production systems on your own.
• Preferred: strong experience with RAG and advanced retrieval, including embeddings, vector and hybrid search, reranking, chunking, grounding and citations, and multi-hop retrieval, Knowledge Graphs or GraphRAG.
• Preferred: familiarity with Agent frameworks such as LangGraph, LangChain, AutoGen and LlamaIndex, and with MCP (Model Context Protocol) for connecting Agents to external tools and systems. Understanding the underlying architecture matters more than knowing one specific framework.
• Preferred: experience designing Agent runtime mechanisms, including human-in-the-loop workflows such as risk classification, approval, interruption and pause/resume, plus context management for long-running workflows, including state persistence, history compression and memory systems.
• Preferred: experience with LangSmith or similar LLM observability and evaluation platforms.
• Preferred: experience with LLM fine-tuning, including LoRA or other parameter-efficient techniques.
• Preferred: experience applying LLM technology to networking, infrastructure, cybersecurity, observability or other complex technical domains.
• Preferred: fluent in English and Chinese, with strong cross-regional communication and collaboration skills.

Conditions

The estimated base salary is CAD $130,000 to CAD $165,000, plus bonus. The final figure may vary with market conditions and individual qualifications assessed during interviews, and the range is a guideline that may change.

The company conducts individual background reviews as part of hiring decisions, in line with applicable law.

It is illegal in Massachusetts to require or give a lie detector test as a condition of employment or continued employment.

Benefits

• Compensation includes cash pay plus a benefits package, including RRSP and medical and dental coverage. Ask the recruiter for details of the Total Rewards approach.