Evaluate AI agents remotely through rubric writing, trace debugging, and stress testing. Outlier offers freelance work for experienced software engineers.
This AI Agent Evaluation Engineer opportunity supports Outlier projects training Large Language Models as proactive, multi-step agents. You will assess system behavior and help improve complex real-world architectural workflows. Candidates need at least two years in backend engineering, AI automation, or complex systems integration. Practical experience with live tools, persistent state, or agent safety analysis is valuable.
Key Responsibilities: Create evaluation rubrics for agent performance
Debug traces from multi-step agent interactions
Stress test autonomous agent behavior
Assess failures across complex system workflows
Requirements: At least two years of relevant engineering experience
Maintain modular, production-grade software systems
Build for live rather than mocked environments
Deliver transparent, detailed technical feedback
Recognize indirect prompt injection risks
Use agent evaluation, systems integration, and technical feedback skills to help Outlier improve advanced generative AI systems.

#J-18808-Ljbffr

📌 AI Agent Evaluation Engineer (Ontario)
🏢 Outlier
📍 Ontario