Transform AI evaluation at Dialpad, a leading customer experience platform, as an AI Evaluation Engineer in Vancouver. You will focus on validation strategies, LLM metrics development, and impactful data analysis.
In this vital role, you will co-own evaluation initiatives for Dialpad's Agentic AI systems, contributing to product readiness and quality assurance. Responsibilities include designing validation strategies across various workflows, executing regression evaluations, and ensuring calibration datasets are well-managed. Your analytical skills will drive improvements in AI products while collaborating with cross-functional teams.
Key Responsibilities:
• Design validation strategies for NLP and speech workflows
• Execute regression evaluations and A/B comparisons




• Co-own LLM metric development and calibration
• Create and monitor data annotation jobs
• Investigate and triage bugs and issues
Requirements:
• Bachelor’s or Master’s in Computer Science or related field
• 3+ years in QA, test engineering, or model evaluation
• Experience with complex AI systems
• Robust analytical skills with cross-functional teams
• Proficiency in designing structured test strategies
Contribute to creative AI solutions at Dialpad, enhancing customer experience through thorough evaluation processes.
J-18808-Ljbffr

📌 Ai Evaluation Engineer In Vancouver
🏢 Dialpad Japan
📍 Vancouver