Evaluate LLM agents with Outlier using technical feedback, agent tracing, and stress testing. Work remotely as a freelance contributor on flexible projects.
Outlier helps innovative companies improve AI agents by providing human feedback. As an LLM Agent Evaluation Engineer, you will support systems designed for proactive, multi-step workflows. The work suits software developers and orchestration specialists with production experience, attention to detail, and complex integration skills.
Key Responsibilities:
- Debug agent traces and investigate system behavior
- Write rubrics for evaluating agent performance
- Stress test autonomous agents against complex workflows
Requirements:
- Two or more years of relevant qualified experience
- Experience building production-grade modular software
- Practical multi-turn system interaction experience
- Ability to deliver high-density technical feedback
- Experience identifying privacy leaks and prompt injections
Use LLM evaluation, debugging, and systems expertise to improve autonomous agent performance with Outlier.