Evaluate autonomous AI agents with Outlier using technical feedback, agent debugging, and complex systems integration. This freelance work is remote.
Outlier helps cutting-edge companies improve AI agents by collecting human feedback. As an engineer, you will help train large language models to operate as proactive, multi-step systems. Projects involve architectural workflows, live tools, and system behaviors requiring careful analysis.
Key Responsibilities:
- Debug agent traces and investigate system behavior
- Write clear evaluation rubrics for agent outputs
- Stress test agents against complex interactions
- Identify privacy leaks and authority escalation failures
- Review indirect prompt injection risks
Requirements:
- At least two years of relevant professional experience
- Backend engineering, AI automation, or integration background
- Maintain production-grade modular software systems
- Handle persistent state and multi-turn interactions
- Provide precise technical feedback on complex behaviors
Use agent evaluation, debugging, and security analysis skills to help Outlier improve autonomous AI systems like OpenClaw.