Evaluate AI-generated code at Turing using Python, TypeScript, or another production language. Work with coding agents across real-world repositories and assess technical quality.

This independent contractor role suits experienced, hands-on software engineers with five or more years of experience. You will review agent behavior, compare solutions, identify failure modes, and create evaluation signals that improve coding models. The work includes collaboration with researchers and engineers.

Key Responsibilities:

- Evaluate AI-generated code across real-world software repositories
- Review agent behavior, tool usage, and code changes
- Identify technical errors and recurring model failure modes
- Create rubrics and evaluation criteria for coding tasks




- Produce preference data and actionable research findings

Requirements:

- Five or more years of hands-on software engineering experience
- Robust Python, TypeScript, JavaScript, Go, or equivalent proficiency
- Experience working in substantial real-world codebases
- Strong code-review skills and technical judgment
- Experience using modern LLMs or AI coding tools

Apply software engineering judgment, code review, and clear communication to improve Turing’s AI coding model evaluations.

📌 AI Coding Model Evaluation Engineer (Montreal)
🏢 Turing
📍 Montreal