Evaluate LLM agents using technical feedback, trace debugging, and stress testing while working remotely as a freelance Outlier contributor.
Outlier supports AI organizations training Large Language Models as proactive, multi-step agents. This work focuses on teaching systems to coordinate complex, real-world architectural workflows through human feedback. The opportunity suits engineers with two or more years in backend engineering, AI automation, or complex systems integration.
Key Responsibilities:
- Create explicit evaluation rubrics for agent performance
- Inspect and debug agent traces
- Stress-test agents against complex scenarios
- Identify subtle system behavior failures
Requirements:
- Two or more years of relevant professional experience
- Production-grade modular software development experience
- Live tool and multi-turn interaction experience
- Strong attention to detail
- Clear, high-density technical communication
Use LLM evaluation, agent debugging, and systems expertise to help improve autonomous software through Outlier.