Evaluate AI agents through rubrics, trace debugging, and stress testing while working remotely for Outlier. Use GPT, Claude, and other leading LLMs.

Outlier helps innovative companies improve AI agents with detailed human feedback. This freelance engineering work supports systems designed as proactive, multi-step agents coordinating complex real-world workflows. The work suits professionals with backend engineering, AI automation, or complex systems integration experience, especially those comfortable reviewing technical behavior.

Key Responsibilities:

- Create evaluation rubrics for agent performance
- Investigate and debug agent traces
- Stress-test agents in live environments
- Assess complex system behaviors with technical precision

Requirements:

- Two or more years of relevant engineering experience
- Production-grade software development with modular separation
- Practical experience with non-mocked systems
- Robust attention to detail and technical feedback skills
- Experience handling multi-turn interactions

Bring backend engineering and AI evaluation expertise to Outlier projects improving autonomous agent reliability and real-world performance.

📌 AI Agent Evaluation Engineer (Vancouver)
🏢 Outlier
📍 Vancouver