Evaluate AI agents remotely using debugging, stress testing, and technical analysis for Outlier’s large language model projects.
Outlier collaborates with AI organizations to improve autonomous systems through human feedback. This freelance work focuses on teaching agents to design, coordinate, and optimize complex architectural workflows. Candidates need software experience with live environments and multi-turn interactions.
Key Responsibilities: Develop evaluation rubrics for agent performance
Inspect and debug detailed agent traces
Stress test autonomous agents across workflows
Assess complex system behaviors with exact feedback
Requirements: Two or more years of relevant engineering experience
Production-grade software development with modular separation
Experience with live, non-mocked environments
Comfort handling multi-turn system interactions
Strong attention to technical detail and failure modes
Use AI evaluation, trace debugging, and systems expertise to improve autonomous agent quality through Outlier projects.

#J-18808-Ljbffr

📌 AI Agent Evaluation Engineer (Alberta)
🏢 Outlier
📍 Alberta