Evaluate autonomous agents remotely with technical feedback, agent trace debugging, and stress testing for Outlier’s AI projects. Adaptable scheduling is available.
Outlier helps innovative companies improve AI agents by providing human feedback on complex behaviors. As an autonomous agent evaluation engineer, you will assess proactive, multi-step systems. Your work supports large language models learning to design, coordinate, and optimize architectural workflows.
Key Responsibilities: Develop rubrics for assessing agent outputs
Investigate failures within agent traces
Test agents against demanding real-world workflows
Requirements: Two or more years of relevant experience
Experience building production-grade modular software
Background in backend engineering or AI automation
Experience with live multi-turn system interactions
Detailed feedback on complex system behaviors
Apply evaluation, debugging, and systems expertise while helping Outlier improve proactive agents and advanced generative systems.