Evaluate autonomous AI systems through agent tracing, stress testing, and rubric writing. This remote freelance work supports Outlier projects for advanced generative systems.
Outlier collaborates with leading AI organizations to improve large language models as proactive, multi-step agents. Applicants need two or more years in backend engineering, AI automation, or complex systems integration. The work centers on production-grade software, live tools, persistent state, and multi-turn interactions.
Key Responsibilities:
- Create evaluation rubrics for complex agent tasks
- Investigate failures within agent execution traces
- Stress test autonomous agents against demanding workflows
- Assess system behaviors with high-density technical feedback
Requirements:
- Two or more years of applicable engineering experience
- Maintain modular, production-grade software architectures
- Build for live environments rather than mocked systems
- Manage multi-turn system interactions carefully
- Identify privacy leaks, authority escalation, or prompt injections
Bring software engineering and AI automation experience to rigorous agent evaluation work involving OpenClaw and real-world workflows.