21 Nov
|
Equifax Canada
|
Winnipeg
21 Nov
Equifax Canada
Winnipeg
Build reliable autonomous workflows at Equifax in Toronto using Python, LLM evaluation, and observability for production AI systems.This newly created full time role focuses on Agentic AI Evaluation and Tuning Engineering. You will test multi-agent architectures, audit decision-making logic, and optimize tool use. The work connects automated testing, prompt tuning, orchestration frameworks, and production guardrails.Key Responsibilities:
Curate Golden Sets of documents, queries, and agent trajectoriesAutomate continuous evaluation pipelines for accuracy, latency, and token spendAudit multi-step reasoning and identify deviations from intended logicOptimize prompts, context windows, and few-shot examplesMonitor deployed agents for drift, injections, loops,
and hallucinationsRequirements:
Three-plus years in quality engineering, test automation, or data engineeringRecent focus on LLM testing, prompt tuning, or orchestrationHands-on LangGraph, ADKs, or specialized internal SDK experiencePython-heavy automated testing and tracing experienceUnderstanding of JSON schemas, function calling, and structured outputsApply Agentic AI evaluation, Python automation, and observability expertise to strengthen Equifax autonomous systems in Toronto.
#J-18808-Ljbffr
📌 Agentic Ai Evaluation Engineer (Winnipeg)
🏢 Equifax Canada
📍 Winnipeg