Evaluate AI agents for Outlier through rubric writing, trace debugging, and stress testing. Work remotely and freelance on a schedule you choose.
Outlier helps innovative companies improve AI agents by supplying human feedback for large language model development. This chance suits software developers and orchestration specialists with experience in complex systems. Projects teach generative systems to coordinate real-world, multi-step workflows through detailed evaluation.
Key Responsibilities: Create rubrics that assess agent performance and behavior
Inspect and debug agent traces across interactions
Stress test autonomous agents for subtle failures
Evaluate systems operating with live tools and data
Requirements: Two or more years of relevant technical experience
Maintain modular, production-grade software services
Build for live environments rather than mocked systems
Give high-density technical feedback on complex behaviors
Identify privacy leaks and indirect prompt injections
Use evaluation, debugging, and security-focused analysis to improve Outlier’s next generation of autonomous AI agents.

#J-18808-Ljbffr

📌 AI Agent Evaluation Engineer (Alberta)
🏢 Outlier
📍 Alberta