Evaluate autonomous AI systems through agent tracing, stress testing, and rubric writing. This remote freelance work supports Outlier projects for advanced generative systems.
Outlier collaborates with leading AI organizations to improve large language models as proactive, multi-step agents. Applicants need two or more years in backend engineering, AI automation, or complex systems integration. The work centers on production-grade software, live tools, persistent state, and multi-turn interactions.
Key Responsibilities: Create evaluation rubrics for complex agent tasks
Investigate failures within agent execution traces
Stress test autonomous agents against demanding workflows
Assess system behaviors with high-density technical feedback
Requirements: Two or more years of applicable engineering experience
Maintain modular, production-grade software architectures
Build for live environments rather than mocked systems
Manage multi-turn system interactions carefully
Identify privacy leaks, authority escalation, or prompt injections
Bring software engineering and AI automation experience to rigorous agent evaluation work involving OpenClaw and real-world workflows.

#J-18808-Ljbffr

📌 Autonomous Agent Evaluation Engineer (Alberta)
🏢 Outlier
📍 Alberta