Evaluate advanced AI agents remotely with Outlier, focusing on technical feedback, agent traces, and real-world system behavior. Work freelance.

Outlier helps creative companies improve AI agents through human feedback on large language model projects. This contract opportunity is designed for software developers and orchestration experts with two or more years’ experience. You will assess systems learning proactive, multi-step behavior across complex architectural workflows.

Key Responsibilities:

- Debug traces across multi-turn agent interactions
- Write rubrics evaluating agent outputs and behavior
- Stress-test agents in live environments

Requirements:

- Two-plus years in backend engineering, AI automation, or systems integration
- Experience building production-grade software with modular separation
- Practical experience handling live, non-mocked systems
- Strong attention to detail and technical communication
- Ability to identify privacy leaks and prompt injections

Use agent evaluation, debugging, and technical feedback skills to help Outlier improve autonomous AI systems.

📌 Remote AI Agent Evaluation Engineer (Toronto)
🏢 Outlier
📍 Toronto