Evaluate autonomous AI agents remotely using technical feedback, trace debugging, and stress testing for Outlier’s generative AI projects.
Outlier collaborates with AI organizations to improve large language models through human feedback. This freelance work focuses on teaching systems to design, coordinate, and optimize real-world architectural workflows. You will assess complex behaviors while working with live, multi-turn agent interactions.
Key Responsibilities: Develop rubrics for evaluating agent performance
Inspect and debug detailed agent traces
Stress test autonomous agents across workflows
Identify privacy leaks and authority escalation risks
Detect indirect prompt injection failures
Requirements: Two or more years of relevant professional experience
Experience building production-grade modular software systems
Practical live-workplace development beyond mocked systems
Strong attention to detail and technical communication
Agent integrations with Supabase, Gmail, or APIs welcomed
Use agent evaluation, systems integration, and failure analysis expertise to improve OpenClaw-like autonomous systems through Outlier.

#J-18808-Ljbffr

📌 AI Agent Evaluation Engineer (Alberta)
🏢 Outlier
📍 Alberta