Evaluate AI-generated code with Python or TypeScript expertise as an independent contractor supporting Turing’s model improvement work.

This experienced software engineering role focuses on reviewing coding agents across real-world repositories. You will assess correctness, robustness, maintainability, and agent behavior, then create evaluation signals that improve coding models. The work includes collaboration with researchers and engineers.

Key Responsibilities:

- Evaluate AI-generated code across real-world software repositories
- Review agent behavior, tool usage, and code changes
- Identify technical errors and recurring model failure modes
- Create rubrics and evaluation criteria for coding tasks
- Produce preference data and actionable findings

Requirements:

- Five or more years of software engineering experience
- Strong Python, TypeScript, JavaScript, Go, or production language proficiency
- Experience with substantial real-world codebases
- Strong code-review skills and technical judgment
- Experience using up-to-date LLMs or AI coding tools

Apply software engineering judgment, code review, and written communication to improve Turing’s AI coding models.

📌 AI Coding Model Evaluation Engineer (Toronto)
🏢 TURING ENTERPRISES
📍 Toronto