Contribute to the forefront of AI solutions with Dialpad as an AI Evaluation Engineer in Vancouver. Focus on LLM metric development and data analysis to ensure high-quality customer interactions.

Your role will be key in the AI Evaluation team's endeavors to maintain and improve Dialpad's Agentic AI systems’ performance. You will design evaluation strategies, execute tests, and conduct thorough error analyses to support the release-readiness of products. Collaborating with engineering and QA teams, your work will greatly influence operational success in AI technology. Key Responsibilities:
Design and implement validation strategies for AI workflows
Conduct regression evaluations and A/B comparisons
Co-own metric development and prompt refinement




Create and manage data annotation jobs
Investigate issues for engineering escalations or analysis Requirements:
Bachelor’s or Master’s degree in a relevant field
3+ years of experience in QA or model evaluation
Comfort with complex AI systems and test datasets
Solid analytical mindset for quality pattern identification
Experience working with technical cross-functional teams Be part of Dialpad’s mission to revolutionize customer experience through cutting-edge AI technologies.

📌 Ai Evaluation Engineer At Dialpad Vancouver
🏢 Dialpad Japan
📍 Vancouver