Evaluate AI agents through rubrics, trace debugging, and stress testing while working remotely for Outlier. Use GPT, Claude, and other leading LLMs.

Outlier helps cutting-edge companies improve AI agents with detailed human feedback. This freelance engineering work supports systems designed as proactive, multi-step agents coordinating complex real-world workflows. The work suits professionals with backend engineering, AI automation, or complex systems integration experience, especially those comfortable reviewing technical behavior.

Key Responsibilities:

- Create evaluation rubrics for agent performance
- Investigate and debug agent traces
- Stress-test agents in live environments
- Assess complex system behaviors with technical precision

Requirements:

- Two or more years of relevant engineering experience
- Production-grade software development with modular separation
- Practical experience with non-mocked systems
- Strong attention to detail and technical feedback skills
- Experience handling multi-turn interactions

Bring backend engineering and AI evaluation expertise to Outlier projects improving autonomous agent reliability and real-world performance.

📌 AI Agent Evaluation Engineer (Victoria)
🏢 Outlier
📍 Victoria