Evaluate autonomous agents remotely with technical feedback, agent trace debugging, and stress testing for Outlier’s AI projects. Flexible scheduling is available.

Outlier helps cutting-edge companies improve AI agents by providing human feedback on complex behaviors. As an autonomous agent evaluation engineer, you will assess proactive, multi-step systems. Your work supports large language models learning to design, coordinate, and optimize architectural workflows.

Key Responsibilities:

- Develop rubrics for assessing agent outputs
- Investigate failures within agent traces
- Test agents against demanding real-world workflows

Requirements:

- Two or more years of relevant experience
- Experience building production-grade modular software
- Background in backend engineering or AI automation
- Experience with live multi-turn system interactions
- Detailed feedback on complex system behaviors

Apply evaluation, debugging, and systems expertise while helping Outlier improve proactive agents and advanced generative systems.

#J-18808-Ljbffr

📌 Autonomous Agent Evaluation Engineer (Toronto)
🏢 Outlier
📍 Toronto