Agentic AI Evaluation Engineer (Toronto)

Agentic AI Evaluation Engineer (Toronto)

22 Nov
|
Equifax Canada
|
Toronto

22 Nov

Equifax Canada

Toronto

Build reliable autonomous workflows at Equifax in Toronto using Python, LLM evaluation, and observability for production AI systems.

This newly created full time role focuses on Agentic AI Evaluation and Tuning Engineering. You will test multi-agent architectures, audit decision-making logic, and optimize tool use. The work connects automated testing, prompt tuning, orchestration frameworks, and production guardrails.

Key Responsibilities:

- Curate Golden Sets of documents, queries, and agent trajectories
- Automate continuous evaluation pipelines for accuracy, latency, and token spend
- Audit multi-step reasoning and identify deviations from intended logic
- Optimize prompts, context windows, and few-shot examples
- Monitor deployed agents for drift, injections, loops,



and hallucinations

Requirements:

- Three-plus years in quality engineering, test automation, or data engineering
- Recent focus on LLM testing, prompt tuning, or orchestration
- Hands-on LangGraph, ADKs, or specialized internal SDK experience
- Python-heavy automated testing and tracing experience
- Understanding of JSON schemas, function calling, and structured outputs

Apply Agentic AI evaluation, Python automation, and observability expertise to strengthen Equifax autonomous systems in Toronto.

#J-18808-Ljbffr

📌 Agentic AI Evaluation Engineer (Toronto)
🏢 Equifax Canada
📍 Toronto

Reply to this offer

Impress this employer describing Your skills and abilities, fill out the form below and leave Your personal touch in the presentation letter.

Subscribe to this job alert:

Get the latest job offers by email for: agentic ai evaluation engineer (toronto) / toronto

Subscribe to this job alert:

Get the latest job offers by email for: agentic ai evaluation engineer (toronto) / toronto