22 Nov
|
Equifax
|
Toronto
Build reliable autonomous workflows at Equifax through LLM evaluation, Python automation, and multi-agent system tuning. This newly created position focuses on enterprise-grade AI. As an Agentic AI Evaluation and Tuning Engineer, you will test and optimize intelligent systems navigating complex software environments. Your work combines Golden Set curation, automated evaluation pipelines, prompt refinement, and tool-calling optimization. You will also investigate agent trajectories, asynchronous failures, and production behavior while collaborating with AI Solution Leads and AI Agent Developers. The position supports reliable, fact-based autonomous decision-making across deployed workflows. Key Responsibilities:
Curate and augment Golden Sets for agent testing Design continuous evaluations for accuracy, latency,
and token spend Audit multi-step reasoning trajectories and deviations Optimize prompts, context windows, and few-shot examples Monitor drift, prompt injections, loops, and hallucinations Requirements:
Three-plus years in quality, automation, data, or ML engineering Recent focus on LLM testing, prompt tuning, or orchestration Hands-on LangGraph, ADKs, or internal SDK experience JSON schema and function-calling expertise Python-heavy automation and asynchronous debugging experience Apply agent evaluation, observability, Python automation, and structured tool calling to strengthen Equifax autonomous systems.
📌 Agentic AI Evaluation Engineer (Toronto)
🏢 Equifax
📍 Toronto