Evaluate autonomous agents with rubric writing and trace debugging skills. This freelance opportunity is remote, flexible, and available through Outlier.

Outlier helps cutting-edge companies improve AI agents by providing human feedback. The work focuses on teaching large language models to design, coordinate, and optimize complex real-world workflows. Contributors assess agent behavior, test systems, and explain subtle failures with high-density technical feedback.

Key Responsibilities

- Create evaluation rubrics for agent performance
- Investigate and debug detailed agent traces
- Stress test agents against complex workflows
- Assess behavior across live system interactions

Requirements:
- Two or more years of relevant experience
- Complex systems integration or AI automation background
- Ability to build maintainable production-grade software
- Experience handling persistent state and session discovery
- Strong attention to technical behavioral details

Apply evaluation, debugging, and systems expertise to help Outlier improve proactive multi-step AI agents like OpenClaw.

📌 Autonomous Agent Evaluation Engineer (Montreal)
🏢 Outlier
📍 Montreal