Embark on an impactful journey at Dialpad as an AI Evaluation Specialist, focusing on agentic AI systems in Vancouver.

Lead the way in LLM-judge metric calibration and robust validation strategies.

As a key figure in our AI Evaluation team, you will conduct structured error analysis and design workflows for NLP and speech technologies.

Your expertise in QA and test engineering will drive essential evaluations for our AI offerings.

This role encompasses collaboration with applied science and engineering teams to prepare AI solutions for the market.

Key Responsibilities:
• Execute validation for agentic AI, NLP, and speech workflows
• Enhance regression evaluations, A/B comparisons, and analyses
• Co-develop LLM-judge metrics and refine prompts




• Create and monitor evaluation datasets for accuracy
• Triage AI issues and recommend solutions or tests

Requirements:
• Bachelor’s or Master’s degree in a tech-related field
• 3+ years of experience in QA or quality assurance
• Familiarity with NLP, speech, and AI system evaluations
• Solid analytical capabilities for failure investigation
• Proven skills in technical collaboration and reporting

Leverage your expertise at Dialpad to shape the future of AI-driven solutions in customer experience.
J-18808-Ljbffr

Required Skill Profession

Engineering

📌 Ai Evaluation Specialist At Dialpad Kitchener
🏢 Doist
📍 Kitchener