Contract Duration: May 4, 2026 to November 4, 2026OverviewWe are seeking an experienced AI / ML QA Engineer to support testing and validation of advanced machine learning and generative AI systems. This role focuses on ensuring model quality, reliability, and responsible AI practices across a range of applications including NLP, classification, recommendation systems, and large language models.Key ResponsibilitiesFunctional and Model TestingDesign and execute test cases for AI and ML models including NLP, classification, recommendation, and generative AI systemsValidate outputs for accuracy, consistency, and expected behavior across diverse datasets and inputsPerform regression testing following model updates, retraining cycles, or data changesPrompt and LLM TestingDevelop and maintain prompt test libraries for LLM based featuresEvaluate responses for relevance, tone, factual accuracy, and alignment with business rulesIdentify hallucinations, response drift, and edge case failuresAI Safety and Red TeamingConduct adversarial testing to uncover vulnerabilities such as prompt injection and jailbreak scenariosAssess outputs for bias, fairness,



and compliance with responsible AI standardsDocument risks and collaborate with model teams on remediation strategiesTest Automation and ToolingBuild and maintain automated test suites using Python based frameworksTrack and report quality metrics, model performance benchmarks, and defect trendsCollaboration and DocumentationWork closely with data scientists, ML engineers, and product teamsDevelop explicit test plans, defect reports, and evaluation summariesContribute to QA standards and best practices for AI systemsRequired SkillsHands on experience testing AI and ML models including NLP, classification, and generative AIStrong experience with LLM testing including prompt engineering and evaluation of hallucinations, bias, and response qualityProficiency in Python and test automation frameworksExperience working with CI CD pipelinesSolid understanding of regression testingNice to HaveExperience in banking or financial servicesExposure to AI safety and red teaming techniques such as prompt injection and jailbreak testingKnowledge of responsible AI and model governance principlesFamiliarity with LLM evaluation tools and frameworksExperience testing recommendation or ranking systemsExposure to ML pipelines and MLOps workflowsUnderstanding of bias and fairness metricsNote: We use AI tools to: obtain basic information, detect plagiarism, false employment history or references, categorize your skills, and do an initial match with job posting.#J-18808-Ljbffr

📌 Ai / Ml Qa Engineer (Toronto)
🏢 Infotek Consulting
📍 Toronto