Agentic AI Evaluation Engineer (Ontario)

Agentic AI Evaluation Engineer (Ontario)

21 Nov
|
Equifax
|
Ontario

21 Nov

Equifax

Ontario

Build reliable autonomous workflows at Equifax through LLM evaluation, Python automation, and multi-agent system tuning. This newly created position focuses on enterprise-grade AI.
As an Agentic AI Evaluation and Tuning Engineer, you will test and optimize intelligent systems navigating complex software environments. Your work combines Golden Set curation, automated evaluation pipelines, prompt refinement, and tool-calling optimization. You will also investigate agent trajectories, asynchronous failures, and production behavior while collaborating with AI Solution Leads and AI Agent Developers. The position supports reliable, fact-based autonomous decision-making across deployed workflows.
Key Responsibilities: Curate and augment Golden Sets for agent testing
Design continuous evaluations for accuracy, latency, and token spend




Audit multi-step reasoning trajectories and deviations
Optimize prompts, context windows, and few-shot examples
Monitor drift, prompt injections, loops, and hallucinations
Requirements: Three-plus years in quality, automation, data, or ML engineering
Recent focus on LLM testing, prompt tuning, or orchestration
Hands-on LangGraph, ADKs, or internal SDK experience
JSON schema and function-calling expertise
Python-heavy automation and asynchronous debugging experience
Apply agent evaluation, observability, Python automation, and structured tool calling to strengthen Equifax autonomous systems.

#J-18808-Ljbffr

📌 Agentic AI Evaluation Engineer (Ontario)
🏢 Equifax
📍 Ontario

Reply to this offer

Impress this employer describing Your skills and abilities, fill out the form below and leave Your personal touch in the presentation letter.

Subscribe to this job alert:

Get the latest job offers by email for: agentic ai evaluation engineer (ontario) / ontario

Subscribe to this job alert:

Get the latest job offers by email for: agentic ai evaluation engineer (ontario) / ontario