Join Dialpad as an AI Evaluation Engineer in Vancouver, where you will enhance AI systems that optimize customer communication. Your work will focus on validation strategies and structured error analysis.
This position plays a crucial role within the AI Evaluation team, overseeing evaluation coverage for Dialpad's creative Agentic AI systems. Responsibilities include designing workflows for validation, building regression evaluations, and managing datasets for effective calibration. Your analytical skills will ensure high-quality AI-driven products through collaborative efforts with technical teams.
Key Responsibilities:
• Design validation strategies for agentic workflows
• Build and manage regression evaluations and A/B tests




• Co-own LLM metric calibration and scenario curation
• Configure data annotation jobs effectively
• Investigate and classify bugs during testing
Requirements:
• Bachelor's or Master's in Computer Science or similar
• 3+ years in QA or applied ML quality
• Familiarity with AI systems like NLP or LLM
• Strong analytical and communication skills
• Experience in structured test strategy design
Drive technological advancement in AI evaluations at Dialpad, making a real impact in customer engagement.
#J-18808-Ljbffr

📌 Vancouver AI Evaluation Engineer Role
🏢 Dialpad Japan
📍 Vancouver