Birlasoft . As a QA Engineer, you will be responsible for both functional and AI testing across the full development lifecycle. This position will test standard application features as well as the AI-specific components such as natural language responses, model behaviors, and prompt handling. You will be working in a team and collaborating closely with developers, product managers, and other stakeholders to ensure a high standard of quality for our AI product.
Location: Mississauga, Ontario. HYBRID - 3 days onsite.
Role Responsibilities
Review and analyze project requirements and technical specifications.
Perform manual testing: functional, regression, smoke, and exploratory testing.
Create detailed test plans, test strategies, and comprehensive test cases.
Identify, document, track, and manage bugs to resolution using Azure DevOps.
Validate and evaluate AI model responses across different scenarios and edge cases.
Design test cases that reflect real-world usage patterns for AI chat.
Compare and analyze outputs between different AI models or versions.
Collaborate with developers, product owners, and stakeholders to ensure product quality.
Participate in Agile ceremonies including sprint planning, stand-ups, and retrospectives.
Technical Qualifications
Strong foundation in QA theory and practice.
Experience in writing test plans, test strategies, and test cases.
Skilled in identifying bugs and creating detailed, clear, and concise bug reports.
Experience in testing web applications and APIs.
Familiarity with AI/ML concepts and how they impact QA.
Knowledge of model performance metrics and AI evaluation techniques.
Ability to write test cases for AI model responses, validate outputs, and identify edge case failures.
Understanding of prompt engineering and the unique testing challenges of generative AI systems.
Experience using Azure DevOps for test case management and bug tracking.
Knowledge of Playwright and TypeScript for test automation is considered a plus.
Mobile testing experience is considered a plus.
General Qualifications
Excellent written and verbal English communication skills.
Solid understanding of the Software Development Life Cycle (SDLC) and QA methodologies.
Proven experience working in a Scrum/Agile environment.
Ability to work collaboratively in cross-functional teams.
Background in insurance or other regulated industries.