Join Dialpad as an AI Evaluation Engineer located in Kitchener, Canada, and drive innovation in customer interaction technologies. Focus on evaluation coverage for agentic AI systems through collaboration and expertise.

This pivotal role in the AI Evaluation team will have you co-owning LLM-judge metrics and curating datasets essential for product development. Your experience in QA and model evaluation will be key to structuring test strategies and conducting error analysis. Robust communication and analytical capabilities are a must for effective teamwork.

Key Responsibilities:
• Design validation strategies for AI-driven workflows
• Perform regression evaluations and red teaming analyses
• Manage LLM-judge metric calibration and prompt refinement




• Create and monitor data annotation tasks and schedules
• Analyze bugs and determine follow-up actions

Requirements:
• Bachelor’s or Master’s degree in a relevant field
• At least 3 years in QA or model quality for AI
• Expertise in structured test strategies in AI contexts
• Excellent analytical skills for quality assessment
• Strong communication skills for cross-team collaboration

Bring your expertise in AI evaluation to Dialpad and help redefine customer experience.
#J-18808-Ljbffr

📌 Dialpad AI Evaluation Engineer Opportunity (Ontario)
🏢 Dialpad Japan
📍 Ontario