Join Dialpad as an AI Evaluation Engineer to enhance the performance of Agentic AI systems. This role focuses on validation strategies, metric development, and collaboration with cross-functional teams in Vancouver. As part of the AI Evaluation team, your main responsibility will be owning evaluation coverage for our AI systems.

You will design and execute validation strategies across various workflows, improving regression evaluations and metrics. Your work will ensure our agentic voice and chat solutions are ready for release through thorough analysis and testing. Key Responsibilities:
Design validation strategies for NLP and speech workflows
Execute regression evaluations and A/B comparisons
Develop LLM-judge metrics and calibration processes
Monitor data annotation jobs for accuracy
Collaborate with applied science and engineering teams Requirements:
Bachelor's or Master's in Computer Science or related field
3+ years in QA, test engineering, or model evaluation
Experience with AI-driven product evaluation
Robust analytical skills and documentation
Familiarity with team-oriented technical environments Make your mark at Dialpad by utilizing your QA expertise to innovate AI technology solutions.

📌 Ai Evaluation Engineer At Dialpad Vancouver
🏢 Doist
📍 Vancouver