Evaluate autonomous agents with backend engineering expertise, debugging traces and stress testing systems through Outlier’s remote freelance platform.
Outlier collaborates with leading AI organizations to improve large language models through human feedback. This freelance work centers on teaching proactive, multi-step agents to coordinate complex architectural workflows. Candidates need at least two years in backend engineering, AI automation, or complex systems integration, plus production software experience.
Key Responsibilities: Develop rubrics for evaluating agent responses
Investigate failures within agent traces
Stress test autonomous agent workflows
Assess behavior across live system interactions
Deliver high-density technical feedback
Requirements: Two or more years of relevant engineering experience
Production-grade software development and maintenance experience
Modular separation across parsing, logic, and reporting
Practical experience with non-mocked environments
Ability to identify subtle system behavior failures
Use evaluation engineering, backend development, and detailed feedback to help improve autonomous agents and advanced generative systems.

#J-18808-Ljbffr

📌 Autonomous Agent Evaluation Engineer (New Brunswick)
🏢 Outlier
📍 New Brunswick