You and the process

With a person
Anna, in-house recruiter
9 years hiring engineers
30 minutes with a real recruiter
They read your CV with you, on a call, and say where the offers are being lost.
Didn’t find what you were looking for? Tell us what to build
Aptiv

AI QA Evaluation Engineer

  • Office

Salary

Not stated

AI summary

For members

The whole posting in a few lines. Sign up to read it here and on every role you open.

Sign Up to Read

Description

The company is building and rolling out enterprise AI agents and agentic workflows, and this role sits in the AI Platform team, reporting to the AI Platform Manager.

You will work day to day with business stakeholders and engineers to set up and keep improving how AI agents are tested and evaluated.

You will own the evaluation frameworks and test standards, and keep development and test environments running reliably.

You will also support business users in acceptance testing and check agents they build before they are published, so they meet enterprise standards, governance rules and quality bars.

Responsibilities

  • Design and roll out testing and evaluation frameworks for AI agents and agentic workflows, covering functional quality, groundedness, safety, reliability and performance.
  • Write and run test plans, test cases, evaluation criteria, test data, datasets and benchmark scenarios so that results are consistent and repeatable.
  • Work with DevOps Agent Engineers and Release and Operations Engineers to validate AI solutions across build, integration, pre-production and deployment.
  • Work with AI Architects to make sure evaluation methods fit the solution design, risk profile, enterprise standards and responsible AI requirements.
  • Own and maintain development and test environments for AI agents, keeping them stable, available and aligned with enterprise platform standards.
  • Support business users in user acceptance testing and check end-user built agents before publishing, confirming they are fit for purpose and meet quality and governance requirements.
  • Log, track and follow up defects with engineering teams through fixing and retesting, and keep improving test practices, tooling and automation.
  • Work with Cyber Security, Legal and Data Privacy teams so testing covers compliance, security and responsible AI, and keep clear records of test results, quality metrics and evaluation outcomes.

Requirements

  • A bachelor's degree in Computer Science, Software Engineering, Information Systems or a related field.
  • Experience in software testing, QA engineering or validation within enterprise technology environments.
  • A solid grasp of test design, test execution and defect management.
  • Experience working with development teams in agile or Kanban delivery.
  • Ability to set out structured test approaches for complex systems, including integration and end-to-end scenarios.
  • Experience working across engineering, architecture and control functions in a governed enterprise setting.
  • Strong analytical and problem-solving skills and close attention to detail.
  • Highly desirable: understanding of AI/ML or automation solutions and the particular testing challenges they bring.
  • Preferred: experience with AI evaluation frameworks covering groundedness, safety, hallucination detection and performance testing.
  • Preferred: familiarity with Microsoft Copilot Studio, Azure AI Foundry or similar enterprise AI platforms.
  • Preferred: experience with test automation tools, scripting or CI/CD-integrated testing.
  • Preferred: exposure to data validation, prompt testing or evaluation dataset design.
  • Preferred: experience with enterprise governance, responsible AI, security and compliance requirements.
  • Preferred: experience working in global, matrixed or regulated enterprise environments.

Conditions

The role is based at Grand Canal Dock, Dublin, which is well served by public transport, including DART trains, buses and bike-sharing.

Benefits

  • Personal holidays, healthcare, pension, a tax saver scheme, free onsite breakfast and lunch, and a discounted corporate gym membership.
  • Access to internal and external training, coaching and certifications.
  • Recognition for innovation and excellence.
  • A multicultural workplace with learning and professional growth in an international setting.
  • A team culture that gives everyone room to grow, lead and make an impact, whatever their background.

Where you’d work

From the office

About the company

Aptiv

  • Industry: Mobility

Office in Dublin, Ireland

Also hiring in Gothenburg, Sweden, United States, Canada and 6 more places

3 of their 87 open roles are remote

Your chances

Worth a look before you spend an evening tailoring a CV for it.

  • 19 checks run
  • 1 red flag

Still hiring?

13 checks

1 red flag

How crowded?

6 checks

No red flags

Fits Me

How well does this role fit you?

Answer a few questions or drop your CV, and every role gets a fit score with the reasons, this one first.

  • Your field
  • Level
  • Stack
  • Work model
  • Salary floor
  • Must-haves
Details10 facts · Role, Location, Compensation, Employment, Company
Tech stack
  • CI/CD
Type
Full-time
Industry
Mobility
Specialty
Manual
Region
Europe
Pay period
Annual
Show 4 more factsShow less

Role

Category
QA & Testing
Specialty
Manual
Tech stack
  • CI/CD

Location

Work model
Office
Region
Europe
Office
  • Dublin, Ireland

Compensation

Salary
Salary by agreement
Pay period
Annual

Employment

Type
Full-time

Company

Industry
Mobility

Something wrong with this vacancy?

Similar vacancies

Share this vacancy

What's wrong with it?

The employer never sees who reported.

Reason