Iris's client, one of the world's largest multinational investment banking and financial services corporation is looking to hire a AI Tester. Our direct client is a is a world-leading financial services company, provides consumers, corporations, governments, and institutions with a broad range of financial services and products.

Title: AI Tester

Location: Toronto, O (Hybrid)

Duration: 24 Months

:

- Add a GenAI test specialist to bring deep expertise in testing LLM-powered features. You'll own how we evaluate AI quality, working within the existing test frameworks and CI/CD pipelines rather than building them from scratch.

What You'll Do

- Define the GenAI testing strategy and quality metrics (accuracy, groundedness, hallucination, safety)
- Agentic AI Framework, swarm testing, Agent orchestration – Claude/Devin
- Build LLM evaluation suites, including LLM-as-a-judge, and plug them into our existing automation framework
- Create and maintain golden datasets and adversarial prompt sets
- Test RAG pipelines for retrieval quality, context relevance, and answer faithfulness
- Validate AI agents and tool-calling flows across multi-step scenarios
- Lead red teaming: prompt injection, jailbreaks, bias, toxicity, PII leakage
- Track model and prompt regressions across releases and share findings with ML and product teams




- Upskill the wider QA team on GenAI testing practices

What You Bring

- 3+ years in software testing or QA automation, with working Python skills
- Hands-on experience testing LLM or GenAI applications
- Strong grasp of LLM concepts: prompts, embeddings, context windows, temperature, RAG
- Experience with eval tools such as DeepEval, Ragas, Promptfoo, LangSmith, or TruLens
- Comfort with API testing and working inside an existing CI/CD setup

About Iris Software Inc. Iris Software, Inc. (registered as SSA Infosystems in Canada) is a CMMi Level 5 Recent Jersey based company providing information technology solutions to clients nationwide for over 30 years. Iris has been growing at over 100% annually. In a program sponsored by Price Waterhouse, PNC Bank and Marsh, Iris has been honored for being

- One of New Jersey's Finest 25 companies for the year 2001 and 2002.
- Iris is also ranked 75th among Inc 500 s list of privately held companies for year 2001.
- NJ Technology FAST 50 Company for year 2002.

In a competitive industry, we distinguish ourselves by reliability, technical expertise and a history of successfully completed projects for clients ranging from mid-sized to Fortune 1000 companies.

📌 AI Tester (Toronto)
🏢 Iris Software
📍 Toronto