AI QA & Evaluation Engineer

About the role

As an AI QA & Evaluation Engineer, you’ll define how we measure whether an AI system is working. You’ll build test sets, automate evaluations, and track accuracy, safety and consistency, so quality is proven with evidence before anything reaches a client.

What you’ll do

• Build test sets from real examples and define quality metrics
• Automate evaluations for accuracy, groundedness, safety and consistency
• Test AI workflows end to end, including edge cases and failure modes
• Share clear results and recommendations with the team and clients

What we’re looking for

• Experience in software QA or test automation
• Curiosity about how AI systems behave and how to measure them
• Skills in Python or a similar scripting language
• Experience with LLM evaluation tools or methods is a plus