Evaluate autonomous agents with Outlier through remote freelance work focused on rubrics, trace debugging, and stress testing. Set your schedule within project needs.
This AI Agent Engineer opportunity supports training large language models to function as proactive, multi-step agents. Projects involve designing, coordinating, and optimizing complex real-world architectural workflows. You will provide human feedback on agent behavior while working with live systems and multi-turn interactions. Outlier accepts applicants in Australia, Canada, Finland, France, Italy, Norway, the United Kingdom, and the United States.
Key Responsibilities: Develop evaluation rubrics for agent outputs
Inspect and debug detailed agent traces




Test agents against demanding real-world workflows
Identify subtle failures in system behavior
Deliver clear, high-density technical feedback
Requirements: Two or more years of relevant experience
Experience building production-grade modular software
Backend, AI automation, or complex integration background
Practical live-environment and multi-turn interaction experience
Solid attention to technical detail
Use agent evaluation, systems integration, and technical feedback skills to improve Outlier’s advanced generative AI systems.

#J-18808-Ljbffr

📌 Autonomous Agent Evaluation Engineer (Alberta)
🏢 Outlier
📍 Alberta