Evaluate autonomous agents through rubric writing and trace debugging with Outlier, working remotely on flexible freelance projects.
Outlier helps innovative companies improve AI agents through human feedback. This engineering work supports large language models designed for proactive, multi-step operation. You will assess complex system behavior, identify subtle failures, and contribute technical judgment to agent training projects.
Key Responsibilities: Create rubrics for evaluating agent performance
Debug multi-turn agent traces and interactions
Stress test agents across real-world workflows
Identify privacy leaks and authority escalation
Detect indirect prompt injection failures
Requirements: Two or more years of relevant experience
Solid attention to detail during technical evaluation
Clear, high-density feedback on complex behaviors
Experience handling live, non-mocked system interactions
Backend, automation, or integration engineering background
Use agent evaluation, failure analysis, and technical feedback expertise to improve OpenClaw systems with Outlier.