Evaluate autonomous agents remotely with technical feedback, agent trace debugging, and stress testing for Outlier's AI projects. Flexible scheduling is available.
Outlier helps cutting-edge companies improve AI agents by providing human feedback on complex behaviors. As an autonomous agent evaluation engineer, you will assess proactive, multi-step systems. Your work supports large language models learning to design, coordinate, and optimize architectural workflows.
Key Responsibilities
- Develop rubrics for assessing agent outputs
- Investigate failures within agent traces
- Test agents against demanding real-world workflows
Requirements:
- Two or more years of relevant experience
- Experience building production-grade modular software
- Background in backend engineering or AI automation
- Experience with live multi-turn system interactions
- Detailed feedback on complex system behaviors
Apply evaluation, debugging, and systems expertise while helping Outlier improve proactive agents and advanced generative systems.