- At Dialpad, you’ll be part of a collaborative team working toward our shared mission of making our customers and their employees wildly successful. We believe that every conversation matters, and we’re elevating each one with a platform that drives real-time insights and automation for our customers
- We thrive on continuous evolution, where every employee leverages industry-leading AI to constantly refine our platform and our own skills
- As a Sr. SDET in Agentic QA, you will own the test automation and quality frameworks that support Dialpad’s AI Voice Agent services
- You will develop automated tests for end-to-end product experiences, from frontend UI to backend services to APIs to audio/text interactions
- You will test orchestration flows, agent configuration experiences, and guardian safeguards to create robust automated coverage for functionality, performance, reliability, UX, and more
- In this role, you will develop substantial amounts of automated test infrastructure and partner deeply with the development team to make our rapid-growing AI platform more testable, more stable, and more delightful for customers
- This position is based at one of Dialpad’s Canadian offices and reports to a QA Eng Manager in the United States
- Own end-to-end quality for agentic features and workflows, including strategy, development, execution, and release qualification
- Design and build automation tooling and frameworks for AI/LLM-driven systems, including prompt flows, agent orchestration, and tool integrations
- Develop and maintain evaluation frameworks (evals)



to measure response quality, accuracy, and hallucination rates
- Drive automation coverage (80%+ for critical AI workflows) using deterministic + probabilistic validation approaches
- Integrate AI quality checks into CI/CD pipelines with fast feedback cycles (<15 minutes for PR validation)
- Build tooling for LLM observability and debugging, including prompt tracing and response analysis
- Partner with Applied AI teams on prompt engineering, model selection, and evaluation strategies
- Design and execute performance and load tests for AI services (latency, throughput, cost efficiency)
- Identify and mitigate risks related to hallucinations, bias, safety, and edge cases
- Define and track AI quality KPIs (task success rates, precision/recall, latency, etc.)
- Participate in design and architecture reviews to ensure systems are testable, observable, and resilient
- Mentor engineers and contribute to raising the bar on AI quality engineering practices

Benefits

- Company stock options
- 100% paid medical, dental, and vision plan
- Continued education stipend
- Cell phone, home internet, and gym membership stipend
- Catered lunches, free snacks & drinks
- Work from home opportunities





We seek individuals who not only meet our high standards but go beyond themOur ambition is significant, and achieving it requires a team that operates at the highest level. We look for individuals who are not just ambitious but who also possess the traits that are fundamental to our success: Scrappy, Curious, Optimistic, Persistent, and EmpatheticBackend: Python, Go, Google Cloud Platform, Cloud Run / App Engine, Kubernetes, Datastore, Redis, ElasticSearch5+ years of experience in software engineering or SDET roles with an emphasis on software developmentUnderstanding of non-deterministic systems and probabilistic testing approachesExperience building test frameworks and scalable automation systemsStrong programming skills in Python (preferred), Java, or JavaScriptAI Stack: LLM APIs, LiveKit, prompt orchestration frameworks, evaluation toolingExperience testing distributed, cloud-native SaaS systems and APIsStrong collaboration skills with the ability to work across distributed teams and time zonesBachelor’s degree in Computer Science or equivalent practical experienceHands-on exposure to LLMs or AI/ML systems (e.g., OpenAI, Claude, Gemini, or similar platforms)Frontend: Vue3, ReactFamiliarity with AI evaluation techniques (benchmarking, golden datasets, human-in-the-loop validation)Experience with CI/CD pipelines (e.g., Jenkins, GitHub Actions)Demonstrated proficiency in coding with AI agents to accelerate development and improve code quality

📌 Senior Software Development Engineer in Test (Vancouver)
🏢 Dialpad
📍 Vancouver