Evaluate autonomous agents remotely with technical feedback, agent trace debugging, and stress testing for Outlier’s AI projects. Flexible scheduling is available.
Outlier helps cutting-edge companies improve AI agents by providing human feedback on complex behaviors. As an autonomous agent evaluation engineer, you will assess proactive, multi-step systems. Your work supports large language models learning to design, coordinate, and optimize architectural workflows.
Key Responsibilities: Develop rubrics for assessing agent outputs
Investigate failures within agent traces
Test agents against demanding real-world workflows
Requirements: Two or more years of relevant experience
Experience building production-grade modular software
Background in backend engineering or AI automation
Experience with live multi-turn system interactions
Detailed feedback on complex system behaviors
Apply evaluation, debugging, and systems expertise while helping Outlier improve proactive agents and advanced generative systems.

#J-18808-Ljbffr

📌 Autonomous Agent Evaluation Engineer (New Brunswick)
🏢 Outlier
📍 New Brunswick