21 Nov
|
Equifax
|
Ontario
Build reliable autonomous workflows at Equifax through LLM evaluation, Python automation, and multi-agent system tuning. This newly created position focuses on enterprise-grade AI.
As an Agentic AI Evaluation and Tuning Engineer, you will test and optimize intelligent systems navigating complex software environments. Your work combines Golden Set curation, automated evaluation pipelines, prompt refinement, and tool-calling optimization. You will also investigate agent trajectories, asynchronous failures, and production behavior while collaborating with AI Solution Leads and AI Agent Developers. The position supports reliable, fact-based autonomous decision-making across deployed workflows.
Key Responsibilities: Curate and augment Golden Sets for agent testing
Design continuous evaluations for accuracy, latency, and token spend
Audit multi-step reasoning trajectories and deviations
Optimize prompts, context windows, and few-shot examples
Monitor drift, prompt injections, loops, and hallucinations
Requirements: Three-plus years in quality, automation, data, or ML engineering
Recent focus on LLM testing, prompt tuning, or orchestration
Hands-on LangGraph, ADKs, or internal SDK experience
JSON schema and function-calling expertise
Python-heavy automation and asynchronous debugging experience
Apply agent evaluation, observability, Python automation, and structured tool calling to strengthen Equifax autonomous systems.
#J-18808-Ljbffr
📌 Agentic AI Evaluation Engineer (Ontario)
🏢 Equifax
📍 Ontario