Elevate customer experience with Dialpad as an AI Evaluation Engineer in Kitchener, Canada. Own evaluation strategies for our advanced AI systems while collaborating with cross-functional teams.

In this role, you'll join the AI Evaluation team, focusing on validating agentic AI systems and enhancing model evaluation methods. Your responsibilities will encompass developing LLM-judge metrics, curating datasets, and conducting structured error analyses to ensure product readiness.

Solid analytical skills and the ability to work with complex AI technologies are essential for success.

Key Responsibilities:
Design and execute validation strategies across workflows
Build and enhance regression evaluations and A/B tests
Co-own LLM-judge metric development and calibration




Create and manage data annotation jobs for evaluations
Investigate bugs and determine escalation or analysis needs

Requirements:
Bachelor's or Master's degree in a related field
3+ years in QA or model evaluation for AI products
Experience with structured test strategies and AI systems
Solid analytical skills for quality pattern identification
Effective communication with cross-functional teams

Contribute to creative AI-driven solutions as an AI Evaluation Engineer at Dialpad in Kitchener.
J-18808-Ljbffr

📌 Ai Evaluation Engineer In Kitchener
🏢 Dialpad Japan
📍 Kitchener