Contract Duration: May 4, 2026 to November 4, 2026

Overview
We are seeking an experienced AI / ML QA Engineer to support testing and validation of advanced machine learning and generative AI systems. This role focuses on ensuring model quality, reliability, and responsible AI practices across a range of applications including NLP, classification, recommendation systems, and large language models.

Key Responsibilities
Functional and Model Testing

Design and execute test cases for AI and ML models including NLP, classification, recommendation, and generative AI systems

Validate outputs for accuracy, consistency, and expected behavior across diverse datasets and inputs

Perform regression testing following model updates, retraining cycles, or data changes

Prompt and LLM Testing

Develop and maintain prompt test libraries for LLM based features

Evaluate responses for relevance, tone, factual accuracy, and alignment with business rules

Identify hallucinations, response drift, and edge case failures

AI Safety and Red Teaming

Conduct adversarial testing to uncover vulnerabilities such as prompt injection and jailbreak scenarios

Assess outputs for bias, fairness, and compliance with responsible AI standards

Document risks and collaborate with model teams on remediation strategies

Test Automation and Tooling

Build and maintain automated test suites using Python based frameworks





Track and report quality metrics, model performance benchmarks, and defect trends

Collaboration and Documentation

Work closely with data scientists, ML engineers, and product teams

Develop transparent test plans, defect reports, and evaluation summaries

Contribute to QA standards and best practices for AI systems

Required Skills

Hands on experience testing AI and ML models including NLP, classification, and generative AI

Strong experience with LLM testing including prompt engineering and evaluation of hallucinations, bias, and response quality

Proficiency in Python and test automation frameworks

Experience working with CI CD pipelines

Solid understanding of regression testing

Nice to Have

Experience in banking or financial services

Exposure to AI safety and red teaming techniques such as prompt injection and jailbreak testing

Knowledge of responsible AI and model governance principles

Familiarity with LLM evaluation tools and frameworks

Experience testing recommendation or ranking systems

Exposure to ML pipelines and MLOps workflows

Understanding of bias and fairness metrics

Note: We use AI tools to: obtain basic information, detect plagiarism, false employment history or references, categorize your skills, and do an initial match with job posting.

#J-18808-Ljbffr

📌 AI / ML QA Engineer (Ontario)
🏢 Infotek Consulting
📍 Ontario