22 Nov
|
Equifax
|
Canada
Build reliable autonomous workflows at Equifax through LLM evaluation, Python automation, and multi-agent system tuning. This newly created position focuses on enterprise-grade AI.
As an Agentic AI Evaluation and Tuning Engineer, you will test and optimize intelligent systems navigating complex software environments. Your work combines Golden Set curation, automated evaluation pipelines, prompt refinement, and tool-calling optimization. You will also investigate agent trajectories, asynchronous failures, and production behavior while collaborating with AI Solution Leads and AI Agent Developers. The position supports reliable, fact-based autonomous decision-making across deployed workflows.
Key Responsibilities
Curate and augment Golden Sets for agent testingDesign continuous evaluations for accuracy, latency,
and token spendAudit multi-step reasoning trajectories and deviationsOptimize prompts, context windows, and few-shot examplesMonitor drift, prompt injections, loops, and hallucinationsRequirements:
Three-plus years in quality, automation, data, or ML engineeringRecent focus on LLM testing, prompt tuning, or orchestrationHands-on LangGraph, ADKs, or internal SDK experienceJSON schema and function-calling expertisePython-heavy automation and asynchronous debugging experienceApply agent evaluation, observability, Python automation, and structured tool calling to strengthen Equifax autonomous systems.
📌 Agentic Ai Evaluation Engineer (Canada)
🏢 Equifax
📍 Canada