Intermediate QA Tester – AI-Generated Code Testing –

Position Summary iVedha is seeking an Intermediate QA Tester with hands-on experience testing AI-generated code and LLM-powered features. This role is ideal for a QA professional who has worked alongside AI coding assistants such as GitHub Copilot, Claude and Cursor, and who can verify that AI-written application code, test scripts and automation behave correctly, securely and as the requirements intend.

The successful candidate will work closely with developers, senior QA engineers, business analysts and DevOps teams to test AI-generated features, build and maintain automation, validate APIs and data, and report clear, reproducible defects. This is a hands-on testing and automation role and is not focused on product development, application support or test management leadership.

Key Responsibilities

· Test code produced by AI coding assistants for logic errors, missed edge cases, insecure patterns and invented (hallucinated) APIs or libraries.

· Verify that AI-generated features meet the user story and acceptance criteria, not just the prompt that produced them.

· Review AI-written unit and automation tests to confirm they assert meaningful behaviour and do not pass trivially.

· Design positive, negative, boundary and error-handling test cases for AI-generated functionality.

· Build and maintain UI and API automation using Playwright with Python or Selenium, with reusable fixtures, helpers and assertions.

· Use AI tools to draft test cases and scripts, then review, correct and harden the output before adding it to the framework.

· Investigate failed automation runs using logs, traces and screenshots, and fix locators, assertions and timing issues.

· Automate REST API tests and validate JSON payloads, business rules and 2xx–5xx response codes.

· Write SQL queries to validate data accuracy, completeness and schema consistency.

· Test AI chatbot and LLM features for accuracy, hallucinations, bias, off-topic replies and prompt injection.

· Run automated suites in CI/CD pipelines (GitHub Actions, Azure DevOps or Jenkins) and keep results reliable.

· Log explicit, reproducible defects in Jira with steps, evidence and likely root cause, and take part in daily triage.





· Support UAT by preparing test data and scenarios with business users.

· Contribute to test completion reports used for release go/no-go decisions.

Required Experience

· 3–5 years of hands-on QA experience testing web and API applications.

· At least 1 year testing code or test scripts generated by AI tools, with examples of defects caught in that code.

· Working knowledge of Python or Java, sufficient to read pull requests and spot logic or security issues.

· Experience building and maintaining automation in Playwright or Selenium within a shared framework.

· Experience with REST API testing and JSON validation.

· Experience running automated tests in CI/CD pipelines and using Git daily.

· Solid understanding of functional, integration, regression, smoke, negative and end-to-end testing.

· Experience working in Agile (Scrum) teams using Jira for test cases and defect tracking.

Technical Skills The ideal candidate should possess expertise in:

· AI Coding Assistants (GitHub Copilot, Claude, Cursor, Copilot Studio)

· Testing and Code Review of AI-Generated Code

· LLM and Chatbot Testing (accuracy, hallucination, prompt injection, safety)

· AI Evaluation Tools (Promptfoo, DeepEval)

· AI-Assisted and Visual Test Tools (Applitools, Reflect AI)

· UI Automation (Playwright, Selenium WebDriver, Cucumber/BDD)

· Programming (Python, Java, JavaScript)

· API Testing (Postman, Rest Assured, Karate, Playwright API Request)

· SQL and Data Validation (PostgreSQL, Oracle, MySQL, BigQuery)

· CI/CD Pipelines (GitHub Actions, Azure DevOps, Jenkins)

· Source Control (Git)

· Cloud Fundamentals (Azure, AWS, Google Cloud Platform)

· Containers (Docker, Kubernetes)

· Test and Defect Management (Jira, Zephyr, Confluence, OpenText ALM)

Key Deliverables

· Test cases and test results for AI-generated features in each sprint.





· Reviewed and hardened automation scripts added to the shared Playwright/API framework.

· Reproducible defect reports with evidence and likely root cause for issues found in AI-generated code.

· LLM and chatbot evaluation results measured against agreed accuracy and safety thresholds.

· Reliable automated regression runs in the CI/CD pipeline.

· Data validation results for features that read or write enterprise data.

· UAT test data and scenarios prepared with business users.

· Input to release test completion reports for go/no-go decisions.

Preferred Qualifications

· Experience within Banking, Financial Services, Telecom or other highly regulated industries.

· Experience with self-healing or AI-assisted test automation and agentic AI test workflows.

· Accessibility testing experience (keyboard navigation, screen-reader labels, colour contrast, WCAG).

· Mobile testing on iOS and Android using Appium or Perfecto.

· Familiarity with SAST/DAST results and common security flaws in generated code.

· Basic performance testing with JMeter or Gatling.

· Certifications such as ISTQB Foundation or ISTQB Certified Tester – AI Testing (CT-AI) are highly desirable.

Ideal Candidate Profile The successful candidate will have:

· 3–5 years of hands-on QA and test automation experience.

· Practical experience testing and correcting code produced by AI coding assistants.

· A curious, sceptical mindset that questions AI output rather than accepting it at face value.

· Strong analytical, problem-solving and attention-to-detail skills.

· The ability to write defect reports a developer can act on without a follow-up call.

· Good communication and collaboration skills across development, QA and business teams.

Preferred Industry Background

· Candidates with experience in the following environments are strongly preferred:

· Banking and Financial Services

· Telecommunications

· IT Consulting and Managed Services Firms

· SaaS and Cloud-Based Product Companies

· Teams Adopting AI-Assisted Software Development

Education

· Bachelor’s degree in Computer Science, Information Technology, Engineering or a related field, or equivalent practical experience.

· QA or AI testing certifications are an asset.

.

Work Location: In person

📌 Intermediate QA Tester – AI-Generated Code Testing – Job Description (Toronto)
🏢 iVedha
📍 Toronto