Evaluate autonomous agents through rubric writing and trace debugging with Outlier, working remotely on flexible freelance projects.

Outlier helps cutting-edge companies improve AI agents through human feedback. This engineering work supports large language models designed for proactive, multi-step operation. You will assess complex system behavior, identify subtle failures, and contribute technical judgment to agent training projects.

Key Responsibilities:

- Create rubrics for evaluating agent performance
- Debug multi-turn agent traces and interactions
- Stress test agents across real-world workflows
- Identify privacy leaks and authority escalation
- Detect indirect prompt injection failures

Requirements:

- Two or more years of relevant experience
- Strong attention to detail during technical evaluation
- Clear, high-density feedback on complex behaviors
- Experience handling live, non-mocked system interactions
- Backend, automation, or integration engineering background

Use agent evaluation, failure analysis, and technical feedback expertise to improve OpenClaw systems with Outlier.

📌 AI Agent Evaluation Engineer OpenClaw (Vancouver)
🏢 Outlier
📍 Vancouver