Software Development Engineer in Test – GenAI / LLM

Role Description

Work Arrangement: 1 day onsite per week

Experience Required: 6–8 years

Key Roles & Responsibilities

- Validate chatbot responses for accuracy, relevance, completeness, and consistency.

- Create and execute test scenarios, test cases, and regression test suites.

- Identify hallucinations, incorrect responses, formatting issues, and knowledge gaps.

- Perform chatbot conversation analysis and recommend quality improvements.

- Develop and maintain test automation for chatbot validation where possible.

- Work closely with developers, AI engineers, and content teams to resolve defects.

- Track, document, and report defects, test results, and quality metrics.

- Support UAT, release validation, and production quality monitoring.

Key Skillsets

- 5 years of strong QA SDET test automation experience.

- Experience testing AI, LLM, GenAI, chatbots, virtual assistants, or conversational platforms.





- Knowledge of prompt engineering concepts.

- Experience with Jira, Confluence, GitHub, and Agile delivery.

- Experience with API testing and automation frameworks.

- Solid analytical and troubleshooting skills.

- Experience with Python, Selenium, Playwright, PyTest, or similar tools.

- Strong communication and documentation skills.

Essential Skills

- QA SDET and test automation.

- AI, LLM, GenAI, chatbots, and conversational platforms.

- Prompt engineering concepts.

- API testing and automation frameworks.

- Python, Selenium, Playwright, or PyTest.

- Jira, Confluence, GitHub, and Agile delivery.

- Analytical and troubleshooting skills.

- Communication and documentation skills.

Desirable Skills

- Digital: Python

- Digital: Chatbots / Conversational Agents

- Generative AI – Quality Engineering

📌 Software Development Engineer in Test – GenAI / LLM (Toronto)
🏢 Astra North Infoteck
📍 Toronto