Evaluate autonomous agents with technical feedback, agent-trace debugging, and stress testing through Outlier’s remote freelance platform. Flexible scheduling is available.

This prospect supports AI organizations training large language models to function as proactive, multi-step agents. The work focuses on design, coordination, and optimization of complex real-world architectural workflows. Candidates need two or more years in backend engineering, AI automation, or complex systems integration. Production software experience, modular separation, and live system interactions are central to the qualification profile.

Key Responsibilities:

- Develop clear, high-density evaluation rubrics
- Debug traces from multi-step agent workflows




- Stress test agents with real-world interactions
- Assess agent coordination and reasoned outputs
- Review failures involving indirect prompt injections

Requirements:

- Two or more years of relevant technical experience
- Maintain production-grade software across modular services
- Build for live environments rather than mocked systems
- Integrate agents with Supabase, Gmail, and APIs
- Implement persistent state using MEMORY.md

Help train proactive AI systems by combining agent evaluation, software engineering, live integrations, and precise technical analysis.

📌 Autonomous Agent Evaluation Engineer (Montreal)
🏢 Outlier
📍 Montreal