Evaluate autonomous agents through rubric writing and trace debugging with Outlier, working remotely on versatile freelance projects.
Outlier helps innovative companies improve AI agents through human feedback. This engineering work supports large language models designed for proactive, multi-step operation. You will assess complex system behavior, identify subtle failures, and contribute technical judgment to agent training projects.
Key Responsibilities:
- Create rubrics for evaluating agent performance
- Debug multi-turn agent traces and interactions
- Stress test agents across real-world workflows
- Identify privacy leaks and authority escalation
- Detect indirect prompt injection failures
Requirements:
- Two or more years of relevant experience
- Strong attention to detail during technical evaluation
- Clear, high-density feedback on complex behaviors
- Experience handling live, non-mocked system interactions
- Backend, automation, or integration engineering background
Use agent evaluation, failure analysis, and technical feedback expertise to improve OpenClaw systems with Outlier.