Evaluate autonomous agents with rubric writing and trace debugging skills. This freelance opportunity is remote, flexible, and available through Outlier.
Outlier helps cutting-edge companies improve AI agents by providing human feedback. The work focuses on teaching large language models to design, coordinate, and optimize complex real-world workflows. Contributors assess agent behavior, test systems, and explain subtle failures with high-density technical feedback.
Key Responsibilities: Create evaluation rubrics for agent performance
Investigate and debug detailed agent traces
Stress test agents against complex workflows
Assess behavior across live system interactions
Requirements: Two or more years of relevant experience
Complex systems integration or AI automation background
Ability to build maintainable production-grade software
Experience handling persistent state and session discovery
Strong attention to technical behavioral details
Apply evaluation, debugging, and systems expertise to help Outlier improve proactive multi-step AI agents like OpenClaw.