Senior AI Software Engineer

Description

You will help build, modernize, connect and maintain an on-premises chatbot platform that serves the U.S. Department of State from Sterling, VA.

The work centers on LLM and RAG features, vector databases, document processing and current AI inference tooling, with Python, API integration and containers as the base.

The goal is to move existing AI services toward a scalable containerized setup while adding features that improve speed, reliability, security and user experience.

You will work across the full AI application stack, from backend code and model serving to data handling, testing and deployment.

You will coordinate with AI/ML engineers, data engineers, infrastructure staff, cybersecurity teams and government stakeholders in classified, air-gapped settings.

The company is a global technology and professional services firm serving U.S. government agencies, defense and the intelligence community, with about 26,000 staff in 50+ countries.

Responsibilities

• Build and maintain Python applications, microservices and APIs for the on-prem chatbot.
• Extend and configure OpenWebUI and connect it to enterprise AI services and backend systems.
• Write custom OpenWebUI Functions, Tools and integrations for mission-specific needs.
• Connect approved LLMs to the chatbot through OpenAI-compatible APIs and inference frameworks such as vLLM and Ollama.
• Create reusable components for model access, prompt management, response processing and application workflows.
• Work with AI/ML engineers to raise inference speed, response quality and reliability.
• Support multimodal use cases such as document understanding, optical character recognition and vision-language models.
• Build backend features that support secure enterprise AI workflows.
• Design and tune RAG workflows for enterprise document search and question answering.
• Add document processing tools such as Docling, Tesseract and vision-language models to automated ingestion pipelines.
• Build chunking, metadata extraction, embedding and retrieval workflows.
• Integrate embedding and reranking models to improve retrieval accuracy and relevance.
• Use PostgreSQL and pgvector for vector search, semantic retrieval and document indexing.
• Build APIs and background services for document ingestion and knowledge base management.
• Work with data engineers on accurate storage, indexing and retrieval of enterprise information.
• Create tests and evaluations that measure RAG accuracy, retrieval relevance and answer quality.
• Run and maintain inference services built on vLLM, Ollama and similar model-serving frameworks.
• Deploy and configure approved open-weight LLMs, embedding models, rerankers and vision-language models.
• Build software integrations using OpenAI-compatible REST APIs.
• Help tune context length, GPU memory use, concurrency and serving performance.
• Help move from current inference frameworks to better architectures as technology matures.
• Troubleshoot inference errors, model loading problems and integration failures with AI/ML engineers.
• Build benchmarking and performance testing tools for AI workloads.
• Move chatbot components from traditional deployments into containers.
• Build and maintain container images with Podman or Docker.
• Maintain Podman Compose files that define services, networking, volumes and environment variables.
• Connect OpenWebUI, vLLM, Docling, PostgreSQL, pgvector, PgBouncer and Redis.
• Write scripts that make installation, deployment, upgrades and recovery repeatable.
• Add health checks, readiness checks, logging and service dependency management.
• Troubleshoot container networking, authentication, storage, GPU access and service connectivity.
• Work with infrastructure and cybersecurity teams to make deployments more consistent and secure.
• Research and prototype agentic AI capabilities for future chatbot releases.
• Build controlled tool-calling and multi-step AI workflows.
• Connect approved models to authorized enterprise APIs and services.
• Evaluate AI orchestration, workflow automation and Model Context Protocol (MCP).
• Design secure execution limits, permissions, audit logs and approval steps for AI-driven actions.
• Work with stakeholders to judge new AI capabilities for mission fit, security and feasibility.
• Build safeguards that keep humans in control and block unauthorized AI actions.
• Apply engineering practices such as modular design, code reviews, documentation and version control.
• Write automated unit, integration, regression and performance tests with frameworks such as pytest.
• Write integration tests that check communication between OpenWebUI, inference services, document processing and databases.
• Support CI/CD for disconnected and classified environments.
• Investigate production issues using logs, metrics, traces and system diagnostics.
• Connect logging and monitoring to tools such as Splunk.
• Help fix vulnerabilities, manage dependencies and review software supply chains and container security.
• Write technical documentation, deployment procedures, architecture diagrams and runbooks.
• Take part in Agile work such as sprint planning, design reviews and demos.
• Design software for secure, on-premises, air-gapped government systems.
• Build secure authentication, role-based access, API authorization and data protection.
• Make sure AI services talk only through authorized internal network paths.
• Package software and deploy it offline using approved dependencies, container images and model files.
• Work with cybersecurity staff to resolve findings and support RMF activities.
• Help prepare documentation and evidence for TRB/CCB reviews and Authority to Operate (ATO) processes.
• Follow approved procedures for handling sensitive and classified information.

Requirements

• Minimum Secret clearance, with no sponsorship.
• Must currently hold a Secret clearance and be able to obtain a Top Secret clearance.
• No public trust position is required.
• Must be a US citizen.
• Bachelor's degree in computer science, software engineering, artificial intelligence, computer engineering or a related technical field, or equivalent practical experience.
• 5+ years of professional software engineering experience, including backend application development.
• 7+ years of related experience.
• Strong Python skills and experience shipping production-quality software.
• Experience building RESTful APIs and integrating distributed services.
• Hands-on experience adding LLMs or generative AI to applications.
• Understanding of transformer models, inference APIs, prompt engineering and LLM application development.
• Experience with RAG, including document ingestion, embeddings, vector search and retrieval pipelines.
• Experience with relational databases such as PostgreSQL.
• Familiarity with Podman or Docker.
• Experience with Linux development and deployment.
• Proficiency with Git, automated testing, debugging and development best practices.
• Experience troubleshooting applications across several interconnected services.
• Strong analytical, problem-solving, communication and technical writing skills.
• Ability to work well with cross-functional teams and government stakeholders.
• Preferred: Experience with OpenWebUI, including configuration, integration, custom Functions, Tools or extensions.
• Preferred: Hands-on work with vLLM, Ollama, Hugging Face Transformers or other LLM inference tools.
• Preferred: Experience serving or integrating open-weight LLMs on-premises.
• Preferred: Experience with NVIDIA GPUs, CUDA, GPU memory management and inference optimization.
• Preferred: Experience with pgvector, vector similarity search and semantic retrieval.
• Preferred: Familiarity with embedding models, rerankers and cross-encoder architectures.
• Preferred: Experience with Docling, Tesseract or document intelligence technologies.
• Preferred: Experience running production RAG pipelines and evaluation frameworks.
• Preferred: Familiarity with agentic AI frameworks, tool calling and Model Context Protocol (MCP).
• Preferred: Experience building or maintaining containerized microservices.
• Preferred: Familiarity with Redis, PgBouncer and asynchronous background processing.
• Preferred: Experience with Red Hat Enterprise Linux and Windows Server.
• Preferred: Experience with FastAPI, Pydantic, pytest and related Python tools.
• Preferred: Experience using Splunk for monitoring and troubleshooting.
• Preferred: Familiarity with software supply chain security, vulnerability scanning and container hardening.
• Preferred: Experience supporting federal systems, classified networks or air-gapped deployments.
• Preferred: Knowledge of federal cybersecurity requirements, RMF and ATO processes.

Conditions

Onsite in Sterling, VA.

Scheduled at 40 hours per week.

Travel up to less than 10% of the time.

The likely salary range is $140,821 - $190,523. This is not a guaranteed amount, since pay depends on experience, geographic location and possibly contractual requirements, and it could fall outside this range.

Identity verification using biometrics and AI is part of hiring, and you must be on camera during virtual interviews.

Equal opportunity employer that welcomes individuals with disabilities and protected veterans.

Benefits

• US-based employees get a choice of medical plans, some paired with Health Savings Accounts, plus dental and vision plans.
• A 401(k) that accepts pre-tax and post-tax contributions up to IRS annual limits, with a company match.
• Full flex work weeks where possible.
• Paid time off including vacation, sick and personal time, holidays, paid parental, military, bereavement and jury duty leave.
• Short-term and long-term disability, life insurance, accidental death and dismemberment, personal accident, critical illness, and business travel and accident insurance, either provided or available.