The company is a subsidiary of a larger parent group and runs a gaming brand whose hardware and network services reach more than 100 million people.
It is also an entertainment business that owns well-known intellectual properties and builds the experiences tied to its brand.
This contractor role focuses on keeping agentic AI systems and backend services reliable, resilient and high quality. You will automate testing across AI and LLM-driven applications, APIs, microservices and distributed systems.
The ideal candidate has solid quality engineering and test automation skills, along with hands-on work testing generative AI, LLM-based or agentic AI systems. Those systems can behave non-deterministically, so simple pass or fail checks may not be enough.
Responsibilities
Build and carry out quality engineering strategies and test plans for agentic AI systems, backend services, APIs and service integrations.
Design and upkeep scalable automated test frameworks covering API, functional, integration, contract, regression and end-to-end testing.
Create test approaches for non-deterministic AI behavior and define sound criteria for judging results.
Spot quality risks early and apply risk-based, shift-left testing throughout the development lifecycle.
Add automated tests to CI/CD pipelines and set up useful quality gates.
Work with software engineers, product managers and other quality engineering teams to improve testability, observability and product quality.
Dig into defects and production issues to find root causes, gaps in testing and ways to keep improving quality.
Set quality metrics and report on test coverage, reliability, defects and release readiness.
Test agentic AI workflows, including multi-step execution, tool selection and invocation, task completion and failure handling.
Check AI-generated responses for accuracy, relevance, groundedness, consistency and expected behavior.
Test AI guardrails, unexpected inputs and failure cases, including misuse of tools and prompt-based attacks.
Verify how AI agents interact with APIs, tools, data sources and downstream services.
Build reusable automation, test data and evaluation scenarios for repeatable testing of AI and services.
Validate API contracts, schemas, authentication, business rules, error handling and service integrations.
Investigate failures across distributed systems and tell apart application, integration, environment, automation and AI-related problems.
Take part in defect triage, root-cause analysis, regression testing and release-readiness work.
Requirements
A bachelor's degree in computer science, engineering, information technology or a related field, or equivalent practical experience.
4+ years of experience in software development engineering in test, quality engineering, software engineering or a related area.
Hands-on experience writing automated tests in Python, Java, JavaScript, TypeScript or a similar language.
Experience with functional, integration, regression, contract and end-to-end testing.
Experience adding automated tests to CI/CD pipelines.
Strong analytical, debugging and root-cause analysis skills.
Experience working in an Agile environment and collaborating across engineering teams.
Solid grasp of software testing methods and quality engineering practices.
Strong API and services testing skills, covering REST, JSON, schemas, authentication and error handling.
Experience designing and maintaining automated test frameworks.
Experience with Git and CI/CD workflows.
Ability to build risk-based test strategies and choose the right testing layers.
Experience testing distributed systems and service-to-service integrations.
Hands-on experience testing generative AI, LLM-powered applications or agentic AI systems.
Understanding of how to test non-deterministic AI behavior and evaluate outputs without relying only on exact-match assertions.
Preferred (good to have): experience testing agentic AI workflows, including planning, multi-step execution, tool calling, memory or state management.
Preferred (good to have): experience with AI evaluation methods such as evaluation datasets, semantic evaluation, LLM-as-a-Judge, groundedness, relevance or hallucination detection.
Preferred (good to have): experience with AI evaluation or observability tools such as LangSmith, DeepEval, Ragas, Arize Phoenix, Promptfoo or similar.
Preferred (good to have): experience testing RAG systems or AI agents that work with APIs, databases or external tools.
Preferred (good to have): experience with contract testing tools such as Pact and API specifications such as OpenAPI or Swagger.
Preferred (good to have): experience with event-driven architectures, Kafka, Docker, Kubernetes or cloud platforms such as AWS.
Preferred (good to have): experience with performance, scalability, reliability or resiliency testing.
Preferred (good to have): experience setting QE metrics, quality gates or cross-team testing standards.
Conditions
This is a contract role, paid at an hourly rate.
The estimated base pay range is $58 to $72 USD per hour.
Final base pay depends on factors such as knowledge, skills, experience, location and market benchmarks.
The company does background checks at the offer stage for all new hires, which may include criminal checks for some roles, and this requires processing personal information.
The company works to create an inclusive environment and encourages everyone to apply.
Benefits
Eligible for medical, dental and vision benefit offerings.