Evaluate autonomous AI agents remotely with Outlier, focusing on agent traces, evaluation rubrics, and complex systems integration. Freelance scheduling is flexible.

Outlier provides human feedback that helps leading AI organizations improve large language models. This work centers on proactive, multi-step agents, including systems that coordinate real-world architectural workflows. Your technical judgment will help assess how these systems behave in live environments.

Key Responsibilities:

- Develop rubrics for evaluating agent performance
- Inspect and debug detailed agent traces
- Stress test autonomous agents across scenarios
- Assess complex system behaviors with precision

Requirements:

- Two or more years of relevant skilled experience
- Experience building production-grade modular software
- Background in backend engineering or AI automation
- Practical experience with live, non-mocked systems
- Ability to deliver high-density technical feedback

Bring agent evaluation, debugging, and systems integration skills to Outlier projects training more capable autonomous AI systems.

📌 Autonomous AI Agent Evaluator (Saskatoon)
🏢 Outlier
📍 Saskatoon