14 Nov
|
Digivance Solutions
|
Canada
14 Nov
Digivance Solutions
Canada
Software Engineer – AI Training & Model Evaluation
Compensation: $60–$120/hour equivalent
Location: Global | Fully Remote
Employment Type: Contract
Time Commitment: 15+ hours per week
Schedule: Flexible
About the Role
We are seeking experienced Software Engineers with prior professional experience in AI training, model evaluation, or AI benchmarking to contribute to a project focused on improving advanced AI systems.
In this role, you will leverage your software engineering expertise to design challenging programming tasks, evaluate AI-generated code, identify technical weaknesses, and establish objective standards for high-quality solutions.
This opportunity is best suited for engineers who have previously worked on substantive AI-training projects and have a solid foundation in practical software development.
Key Responsibilities
- Design realistic and challenging software engineering tasks to evaluate advanced AI models.
- Develop detailed task instructions, reproducible environments, reference solutions, and automated tests.
- Review and evaluate AI-generated code for correctness, robustness, efficiency, and completeness.
- Identify technical issues related to implementation, debugging, dependencies, performance, and reproducibility.
- Create objective evaluation criteria and grading standards for technical submissions.
- Assess model performance across different programming and software engineering scenarios.
Required Qualifications
- Prior paid professional experience in AI training,
model evaluation, benchmarking, or improving AI systems.
- Strong hands-on proficiency in Python.
- Professional software engineering experience with at least one additional programming language, such as Java, JavaScript, TypeScript, Rust, C++, C#, or Go.
- Strong understanding of software design, debugging, testing, version control, and dependency management.
- Ability to assess complex codebases and identify technical issues independently.
Public Proof of Technical Skills Applicants should be able to demonstrate their technical expertise through publicly accessible evidence, such as:
- Substantial GitHub or GitLab repositories.
- Meaningful open-source contributions.
- Maintainer or core-contributor experience.
- Competitive programming achievements (Codeforces, ICPC, Olympiads, etc.).
- Publicly developed tools, libraries, or software systems.
- Technical publications, benchmark contributions, or engineering write-ups.
Preferred Qualifications
- Experience creating AI-training tasks, reference solutions, automated tests, graders, or benchmark problems.
- Experience reviewing and evaluating technical submissions from other AI trainers.
- Codeforces rating of approximately 1900+ or equivalent competitive programming achievements.
- Familiarity with Linux, Docker, CI/CD, command-line tools, and sandboxed development environments.
- Experience with long-horizon or agentic coding evaluations.
- Availability to start immediately.
📌 Software Engineer – AI Training & Model Evaluation (Canada)
🏢 Digivance Solutions
📍 Canada