Job Description Software Development Engineer in Test – GenAI / LLM Role Description Work Arrangement: 1 day onsite per week Experience Required: 6–8 years Key Roles & Responsibilities Validate chatbot responses for accuracy, relevance, completeness, and consistency.

Create and execute test scenarios, test cases, and regression test suites.

Identify hallucinations, incorrect responses, formatting issues, and knowledge gaps.

Perform chatbot conversation analysis and recommend quality improvements.

Develop and maintain test automation for chatbot validation where possible.

Work closely with developers, AI engineers, and content teams to resolve defects.

Track, document, and report defects, test results, and quality metrics.

Support UAT, release validation, and production quality monitoring.

Key Skillsets 5 years of solid QA SDET test automation experience.

Experience testing AI, LLM, GenAI, chatbots, virtual assistants, or conversational platforms.





Knowledge of prompt engineering concepts.

Experience with Jira, Confluence, Git

Hub, and Agile delivery.

Experience with API testing and automation frameworks.

Strong analytical and troubleshooting skills.

Experience with Python, Selenium, Playwright, PyTest, or similar tools.

Strong communication and documentation skills.

Essential Skills QA SDET and test automation. AI, LLM, GenAI, chatbots, and conversational platforms.

Prompt engineering concepts. API testing and automation frameworks.

Python, Selenium, Playwright, or PyTest.

Jira, Confluence, Git

Hub, and Agile delivery.

Analytical and troubleshooting skills.

Communication and documentation skills.

Desirable Skills Digital: Python Digital: Chatbots / Conversational Agents Generative AI – Quality Engineering Requirements Java

📌 Software Development Engineer in Test – GenAI / LLM (Toronto)
🏢 Astra North Infoteck
📍 Toronto