Evaluate autonomous agents remotely using production-grade software skills and precise technical feedback. Outlier offers freelance work with adaptable scheduling.

This opportunity supports projects training large language models to function as proactive, multi-step agents. Contributors assess complex system behaviors through rubrics, trace debugging, and stress testing. Experience with backend engineering, AI automation, or systems integration supports the work.

Key Responsibilities:

- Develop rubrics for evaluating agent outputs
- Inspect and debug traces from agent interactions
- Test agents against complex workflow scenarios
- Identify subtle failures in system behavior

Requirements:

- Two or more years of relevant experience
- Production-grade software development with modular separation
- Practical experience with live system interactions
- Clear, detailed feedback on complex technical behaviors
- Experience with multi-stage coordination tasks preferred

Use agent evaluation, systems integration, and technical analysis skills to improve proactive AI systems through Outlier projects.

#J-18808-Ljbffr

📌 Autonomous Agent Evaluation Engineer (Montreal)
🏢 Outlier
📍 Montreal