Evaluate autonomous agents through rubric writing, trace debugging, and stress testing. Outlier offers remote freelance work with flexible scheduling for qualified contributors.
This specialist work supports Outlier projects improving AI agents for leading AI organizations. Contributors help train large language models to operate as proactive, multi-step systems. The work centers on complex architectural workflows, live interactions, and detailed assessment of system behavior.
Key Responsibilities:
- Create evaluation rubrics for agent performance
- Inspect and debug multi-turn agent traces
- Stress test autonomous agents across workflows
- Assess complex behaviors with high-density feedback
Requirements:
- Two or more years of relevant qualified experience
- Backend engineering, AI automation, or complex systems integration background
- Production software development using modular separation
- Practical experience with live, non-mocked systems
- Ability to identify privacy leaks and prompt injections
Bring rigorous evaluation, backend engineering, and autonomous-agent expertise to Outlier’s remote OpenClaw training work.