Evaluate AI-generated code at Turing using Python, TypeScript, or another production language. Work with coding agents across real-world repositories and assess technical quality.

This independent contractor role suits experienced, hands-on software engineers with five or more years of experience. You will review agent behavior, compare solutions, identify failure modes, and create evaluation signals that improve coding models. The work includes collaboration with researchers and engineers.

Key Responsibilities:
Evaluate AI-generated code across real-world software repositories
Review agent behavior, tool usage, and code changes
Identify technical errors and recurring model failure modes
Create rubrics and evaluation criteria for coding tasks




Produce preference data and actionable research findings

Requirements:
Five or more years of hands-on software engineering experience
Robust Python, TypeScript, JavaScript, Go, or equivalent proficiency
Experience working in substantial real-world codebases
Solid code-review skills and technical judgment
Experience using up-to-date LLMs or AI coding tools

Apply software engineering judgment, code review, and clear communication to improve Turing’s AI coding model evaluations.

📌 Ai Coding Model Evaluation Engineer Montreal
🏢 Turing
📍 Montreal