Apply AI automation and systems integration skills to evaluate autonomous agents remotely through Outlier’s flexible freelance platform. Weekly payments available.

As an AI Automation Evaluation Engineer, you will provide human feedback that helps leading AI organizations train Large Language Models. Projects teach systems to coordinate multi-step tasks, manage architectural workflows, and produce reasoned outputs. The work requires careful analysis of live agent behavior.

Key Responsibilities:

- Create rubrics assessing agent outputs and workflows
- Investigate failures within agent traces
- Challenge agents through targeted stress testing

Requirements:

- Two or more years in backend engineering or AI automation
- Experience integrating complex systems with modular services
- Hands-on work with multi-turn system interactions
- Solid attention to detail during technical evaluation
- Ability to deliver high-density technical feedback

Use systems integration, agent evaluation, and technical feedback expertise to support higher-quality autonomous AI development with Outlier.

📌 AI Automation Evaluation Engineer (Edmonton)
🏢 Outlier
📍 Edmonton