Join Dialpad as an AI Evaluation Engineer, directly impacting AI-driven customer experiences in Vancouver. Your role will focus on developing LLM-judge metrics and evaluation strategies for agentic systems.

This position requires a driven individual with a strong background in QA, test engineering, and applied ML quality assurance. You will co-own various aspects of AI evaluation, including validation strategies and error analysis to ensure product readiness. Collaborate across teams to refine our AI systems and support their deployment in a competitive market.

Key Responsibilities:
• Design and implement validation strategies for AI workflows
• Build and refine regression evaluations and red teaming analyses
• Collaborate on LLM-judge metric calibration and prompt strategies




• Manage data annotation jobs for timely evaluation datasets
• Investigate bugs and recommend escalation or further analysis

Requirements:
• Bachelor’s or Master’s degree in Computer Science or similar
• Minimum of 3 years of experience with AI product evaluations
• Proficient with NLP, speech, and AI evaluation datasets
• Robust analytical skills for identifying quality issues
• Effective collaboration with technical teams and stakeholders

Drive AI innovation at Dialpad by applying your skills to evaluate and enhance next-generation customer solutions.
#J-18808-Ljbffr

📌 AI Evaluation Engineer Role at Dialpad (Ontario)
🏢 Doist
📍 Ontario