Job Title: Quality Engineer (QE) / SDET – AI Chatbot Validation

Locations: ~Brampton, Ontario~Scarborough, Ontario~Etobicoke, Ontario~Guleph, Ontario~Hamilton, Ontario~Kitchener, Ontario~Milton, Ontario~North York, Ontario~Ontario Nes,~Oshawa, Ontario~

Long Term Contract

Experience: 6–8 Years

Work Model: Hybrid – 1 day onsite per week

:

We are looking for an experienced Quality Engineer / SDET with strong test automation skills and hands-on experience validating AI-powered chatbots, LLMs, Generative AI applications, or conversational platforms. The ideal candidate will ensure the accuracy, relevance, consistency, and reliability of AI-generated responses while developing and maintaining automated testing solutions.

Key Responsibilities:

- Validate chatbot responses for accuracy, relevance, completeness, and consistency.
- Design, develop, and execute test scenarios, test cases, and regression test suites.
- Identify AI hallucinations, incorrect responses, formatting issues, and knowledge gaps.
- Analyze chatbot conversations and recommend quality improvements.
- Develop and maintain automated test frameworks for chatbot validation.
- Collaborate with developers, AI engineers,



and content teams to investigate and resolve defects.
- Track defects, document test results, and report quality metrics.
- Support UAT, release validation, and production quality monitoring.

Required Skills:

- 5+ years of strong QA / SDET / test automation experience.
- Experience testing AI, LLMs, Generative AI applications, chatbots, virtual assistants, or conversational platforms.
- Understanding of prompt engineering and AI response validation.
- Strong programming and automation skills using Python, Selenium, Playwright, PyTest, or similar tools.
- Experience with API testing and automation frameworks.
- Familiarity with Jira, Confluence, GitHub, and Agile delivery methodologies.
- Solid analytical, troubleshooting, communication, and documentation skills.

Preferred Skills:

- Experience with LLM evaluation, hallucination detection, and conversational AI quality metrics.
- Experience building automated validation and regression testing frameworks for AI applications.
- Knowledge of Generative AI quality engineering practices.

📌 Quality Engineer (QE) / SDET – AI Chatbot Validation (Toronto)
🏢 Realtek Consulting
📍 Toronto