Software Engineer – AI Training & Model Evaluation (Canada)

Software Engineer – AI Training & Model Evaluation (Canada)

14 Nov
|
Digivance Solutions
|
Canada

14 Nov

Digivance Solutions

Canada

Software Engineer – AI Training & Model Evaluation

Compensation: $60–$120/hour equivalent

Location: Global | Fully Remote

Employment Type: Contract

Time Commitment: 15+ hours per week

Schedule: Flexible

About the Role

We are seeking experienced Software Engineers with prior professional experience in AI training, model evaluation, or AI benchmarking to contribute to a project focused on improving advanced AI systems.

In this role, you will leverage your software engineering expertise to design challenging programming tasks, evaluate AI-generated code, identify technical weaknesses, and establish objective standards for high-quality solutions.

This opportunity is best suited for engineers who have previously worked on substantive AI-training projects and have a solid foundation in practical software development.

Key Responsibilities

- Design realistic and challenging software engineering tasks to evaluate advanced AI models.
- Develop detailed task instructions, reproducible environments, reference solutions, and automated tests.
- Review and evaluate AI-generated code for correctness, robustness, efficiency, and completeness.
- Identify technical issues related to implementation, debugging, dependencies, performance, and reproducibility.
- Create objective evaluation criteria and grading standards for technical submissions.
- Assess model performance across different programming and software engineering scenarios.

Required Qualifications
- Prior paid professional experience in AI training,



model evaluation, benchmarking, or improving AI systems.
- Strong hands-on proficiency in Python.
- Professional software engineering experience with at least one additional programming language, such as Java, JavaScript, TypeScript, Rust, C++, C#, or Go.
- Strong understanding of software design, debugging, testing, version control, and dependency management.
- Ability to assess complex codebases and identify technical issues independently.

Public Proof of Technical Skills Applicants should be able to demonstrate their technical expertise through publicly accessible evidence, such as:
- Substantial GitHub or GitLab repositories.
- Meaningful open-source contributions.
- Maintainer or core-contributor experience.
- Competitive programming achievements (Codeforces, ICPC, Olympiads, etc.).
- Publicly developed tools, libraries, or software systems.
- Technical publications, benchmark contributions, or engineering write-ups.

Preferred Qualifications
- Experience creating AI-training tasks, reference solutions, automated tests, graders, or benchmark problems.
- Experience reviewing and evaluating technical submissions from other AI trainers.
- Codeforces rating of approximately 1900+ or equivalent competitive programming achievements.
- Familiarity with Linux, Docker, CI/CD, command-line tools, and sandboxed development environments.
- Experience with long-horizon or agentic coding evaluations.
- Availability to start immediately.

📌 Software Engineer – AI Training & Model Evaluation (Canada)
🏢 Digivance Solutions
📍 Canada

Reply to this offer

Impress this employer describing Your skills and abilities, fill out the form below and leave Your personal touch in the presentation letter.

Subscribe to this job alert:

Get the latest job offers by email for: software engineer – ai training & model evaluation (canada) / canada

Subscribe to this job alert:

Get the latest job offers by email for: software engineer – ai training & model evaluation (canada) / canada