Evaluate LLM agents using technical feedback, trace debugging, and stress testing while working remotely as a freelance Outlier contributor.
Outlier supports AI organizations training Large Language Models as proactive, multi-step agents. This work focuses on teaching systems to coordinate complex, real-world architectural workflows through human feedback. The opportunity suits engineers with two or more years in backend engineering, AI automation, or complex systems integration.
Key Responsibilities: Create explicit evaluation rubrics for agent performance
Inspect and debug agent traces
Stress-test agents against complex scenarios
Identify subtle system behavior failures
Requirements: Two or more years of relevant professional experience
Production-grade modular software development experience
Live tool and multi-turn interaction experience
Strong attention to detail
Clear, high-density technical communication
Use LLM evaluation, agent debugging, and systems expertise to help improve autonomous software through Outlier.