Evaluate autonomous agents remotely with technical feedback, agent trace debugging, and stress testing for Outlier's AI projects. Flexible scheduling is available.
Outlier helps creative companies improve AI agents by providing human feedback on complex behaviors. As an autonomous agent evaluation engineer, you will assess proactive, multi-step systems. Your work supports large language models learning to design, coordinate, and optimize architectural workflows.
Key Responsibilities
- Develop rubrics for assessing agent outputs
- Investigate failures within agent traces
- Test agents against demanding real-world workflows
Requirements:
- Two or more years of relevant experience
- Experience building production-grade modular software
- Background in backend engineering or AI automation
- Experience with live multi-turn system interactions
- Detailed feedback on complex system behaviors
Apply evaluation, debugging, and systems expertise while helping Outlier improve proactive agents and advanced generative systems.
📌 Autonomous Agent Evaluation Engineer (Vancouver)
🏢 Outlier
📍 Vancouver
Reply to this offer
Impress this employer describing Your skills and abilities, fill out the form below and leave Your personal touch in the presentation letter.