Software Development Engineer in Test – GenAI / LLM

Role Description Work Arrangement: 1 day onsite per week Experience Required: 6–8 years Key Roles &

Responsibilities Validate chatbot responses for accuracy, relevance, completeness, and consistency. Create and execute test scenarios, test cases, and regression test suites. Identify hallucinations, incorrect responses, formatting issues, and knowledge gaps.

Perform chatbot conversation analysis and recommend quality improvements. Develop and maintain test automation for chatbot validation where possible. Work closely with developers, AI engineers, and content teams to resolve defects.

Track, document, and report defects, test results, and quality metrics. Support UAT, release validation, and production quality monitoring. Key Skillsets 5 years of strong QA SDET test automation experience.

Experience testing AI, LLM, GenAI, chatbots, virtual assistants, or conversational platforms.



Knowledge of prompt engineering concepts.

Experience with Jira, Confluence, GitHub, and Agile delivery.

Experience with API testing and automation frameworks. Robust analytical and troubleshooting skills.

Experience with Python, Selenium, Playwright, PyTest, or similar tools. Strong communication and documentation skills.

Essential

Skills QA SDET and test automation. AI, LLM, GenAI, chatbots, and conversational platforms. Prompt engineering concepts. API testing and automation frameworks. Python, Selenium, Playwright, or PyTest. Jira, Confluence, GitHub, and Agile delivery. Analytical and troubleshooting skills. Communication and documentation skills.

Desirable Skills Digital: Python Digital: Chatbots / Conversational Agents Generative AI – Quality Engineering

📌 Software Development Engineer in Test - GenAI / LLM (Toronto)
🏢 Astra-North Infoteck Inc. ~ Conquering today’s challenges, achieving tomorrow’s vision!
📍 Toronto