Evaluate AI-generated code at Turing using Python, TypeScript, or another production language. Work with coding agents across real-world repositories and assess technical quality.

This independent contractor role suits experienced, hands-on software engineers with five or more years of experience. You will review agent behavior, compare solutions, identify failure modes, and create evaluation signals that improve coding models. The work includes collaboration with researchers and engineers.

Key Responsibilities

Evaluate AI-generated code across real-world software repositories

Review agent behavior, tool usage, and code changes

Identify technical errors and recurring model failure modes

Create rubrics and evaluation criteria for coding tasks

Produce preference data and actionable research findings

Requirements

Five or more years of hands-on software engineering experience

Solid Python, TypeScript, JavaScript, Go, or equivalent proficiency

Experience working in substantial real-world codebases

Strong code-review skills and technical judgment

Experience using modern LLMs or AI coding tools

Apply software engineering judgment, code review, and clear communication to improve Turing’s AI coding model evaluations.

📌 AI Coding Model Evaluation Engineer (Winnipeg)
🏢 TURING ENTERPRISES
📍 Winnipeg