Agentic Ai Evaluation Engineer (Winnipeg)

Agentic Ai Evaluation Engineer (Winnipeg)

22 Nov
|
Equifax
|
Winnipeg

22 Nov

Equifax

Winnipeg

Build reliable autonomous workflows at Equifax through LLM evaluation, Python automation, and multi-agent system tuning. This newly created position focuses on enterprise-grade AI.
As an Agentic AI Evaluation and Tuning Engineer, you will test and optimize intelligent systems navigating complex software environments. Your work combines Golden Set curation, automated evaluation pipelines, prompt refinement, and tool-calling optimization. You will also investigate agent trajectories, asynchronous failures, and production behavior while collaborating with AI Solution Leads and AI Agent Developers. The position supports reliable, fact-based autonomous decision-making across deployed workflows.

Key Responsibilities

Curate and augment Golden Sets for agent testingDesign continuous evaluations for accuracy, latency,



and token spendAudit multi-step reasoning trajectories and deviationsOptimize prompts, context windows, and few-shot examplesMonitor drift, prompt injections, loops, and hallucinationsRequirements:
Three-plus years in quality, automation, data, or ML engineeringRecent focus on LLM testing, prompt tuning, or orchestrationHands-on LangGraph, ADKs, or internal SDK experienceJSON schema and function-calling expertisePython-heavy automation and asynchronous debugging experienceApply agent evaluation, observability, Python automation, and structured tool calling to strengthen Equifax autonomous systems.
#J-18808-Ljbffr

📌 Agentic Ai Evaluation Engineer (Winnipeg)
🏢 Equifax
📍 Winnipeg

Reply to this offer

Impress this employer describing Your skills and abilities, fill out the form below and leave Your personal touch in the presentation letter.

Subscribe to this job alert:

Get the latest job offers by email for: agentic ai evaluation engineer (winnipeg) / winnipeg

Subscribe to this job alert:

Get the latest job offers by email for: agentic ai evaluation engineer (winnipeg) / winnipeg