Be at the forefront of AI technology with Dialpad as an AI Evaluation Engineer in Kitchener, Canada. Drive validation efforts for our advanced agentic AI systems and enhance customer engagement.
Join Dialpad’s AI Evaluation team to take ownership of evaluation processes for cutting-edge AI solutions. Your role emphasizes the development of metrics, scenario dataset curation, and structured error analysis, supporting team readiness for public releases. You will collaborate with various technical teams and leverage your experience in quality assurance and model evaluation.
Key Responsibilities:
• Execute validation strategies for NLP and speech workflows
• Run regression evaluations and A/B comparison analyses
• Develop calibration methods for LLM-judge metrics




• Oversee and monitor data annotation processes
• Investigate issues and decide on troubleshooting strategies
Requirements:
• Degree in Computer Science or related disciplines
• Minimum 3 years in quality assurance or applied ML
• Familiarity with test strategy design and complex AI
• Analytical skills for detailed investigation and comparison
• Proven collaboration with technical teams for documentation
Join Dialpad to shape the future of AI in customer experience as an AI Evaluation Engineer.
#J-18808-Ljbffr

📌 AI Quality Evaluation Engineer Role (Kitchener)
🏢 Dialpad Japan
📍 Kitchener