Build reliable autonomous AI at Equifax through LLM evaluation, Python automation, and multi-agent workflow testing. This newly created position focuses on production reliability.

As an Agentic AI Evaluation & Tuning Engineer, you will connect LLM capabilities with dependable autonomous execution. You will curate Golden Sets, automate evaluation pipelines, audit reasoning trajectories, and tune prompts. Your work will also address tool calling, RAG pipelines, hallucinations, and guardrails. Collaboration with AI Solution Leads and AI Agent Developers supports robust agent components.

Key Responsibilities:
Curate documents, queries, and expected agent trajectories
Automate evaluations for accuracy, latency, token spend, and reliability
Audit multi-step reasoning and identify logic deviations




Optimize prompts, context windows, and few-shot examples
Monitor drift, injections, loops, and hallucinations

Requirements:
Three or more years in quality, automation, data, or ML engineering
Recent LLM testing, prompt tuning, or orchestration experience
Hands-on LangGraph, ADKs, or internal SDK experience
JSON schema and function-calling expertise
Python-heavy automation and asynchronous debugging experience

Apply analytical evaluation, Python automation, and LLM orchestration expertise to strengthen Equifax autonomous AI reliability.
J-18808-Ljbffr

📌 Agentic Ai Evaluation Engineer Toronto (Canada)
🏢 Equifax
📍 Canada