Evaluate autonomous agents through rubric writing, trace debugging, and stress testing. Work remotely with Outlier as a freelance contributor on flexible schedules.
Outlier helps cutting-edge companies improve AI agents by providing human feedback for advanced generative systems. This work focuses on teaching large language models to coordinate multi-step, real-world architectural workflows. You will assess complex system behavior and deliver high-density technical feedback. The opportunity requires two or more years in backend engineering, AI automation, or complex systems integration. Experience with modular production software, live environments, and multi-turn interactions is required. Applicants may apply from Australia, Canada, Finland, France, Italy, Norway, the United Kingdom, or the United States.
Key Responsibilities:
Develop evaluation rubrics for agent performance
Debug traces from multi-step agent workflows
Stress test autonomous agent behavior
Identify privacy leaks and authority escalation
Review indirect prompt injection failures
Requirements: At least two years of relevant experience
Strong attention to complex system details
Clear, high-density technical feedback skills
Production-grade modular software experience
Live, multi-turn systems experience
Use evaluation, debugging, and systems-integration expertise to strengthen autonomous-agent performance for Outlier’s AI projects.