Evaluate LLM agents with Outlier using technical feedback, agent tracing, and stress testing. Work remotely as a freelance contributor on flexible projects.
Outlier helps innovative companies improve AI agents by providing human feedback. As an LLM Agent Evaluation Engineer, you will support systems designed for proactive, multi-step workflows. The work suits software developers and orchestration specialists with production experience, attention to detail, and complex integration skills.
Key Responsibilities: Debug agent traces and investigate system behavior
Write rubrics for evaluating agent performance
Stress test autonomous agents against complex workflows
Requirements: Two or more years of relevant qualified experience
Experience building production-grade modular software
Practical multi-turn system interaction experience
Ability to deliver high-density technical feedback
Experience identifying privacy leaks and prompt injections
Use LLM evaluation, debugging, and systems expertise to improve autonomous agent performance with Outlier.

#J-18808-Ljbffr

📌 LLM Agent Evaluation Engineer (Alberta)
🏢 Outlier
📍 Alberta