- Role Title: Senior Software Engineer – LLM Evaluation & Repository Validation
- Role Type: Contractor assignment (no medical/paid leave)
- Location: Remote

What does day-to-day look like:

- Analyze and triage GitHub issues across trending open-source libraries.
- Set up and configure code repositories, including Dockerization and environment setup.
- Evaluating unit test coverage and quality.
- Modify and run codebases locally to assess LLM performance in bug-fixing scenarios.
- Collaborate with researchers to design and identify repositories and issues that are challenging for LLMs.
- Opportunities to lead a team of junior engineers to collaborate on projects.

Required Skills:

- Solid experience with at least one of the following languages: Python, JavaScript, Java, Go, Rust, C/C++, C#, or Ruby.
- Proficiency with Git, Docker, and basic software pipeline setup.
- Ability to understand and navigate complex codebases.
- Comfortable running, modifying, and testing real-world projects locally.




- Experience contributing to or evaluating open-source projects is a plus.

Nice to Have:

- Previous participation in LLM research or evaluation projects.
- Experience building or testing developer tools or automation agents.

Offer details:

- Commitment required: 20 hours per week with some overlap with PST
- Employment type: Contractor assignment (no medical/paid leave)
- Duration of contract: 3 months with expected start date as next week

What You’ll Do

- Analyze and triage GitHub issues across trending open-source libraries.
- Set up and configure code repositories, including Dockerization and environment setup.
- Evaluating unit test coverage and quality.
- Modify and run codebases locally to assess LLM performance in bug-fixing scenarios.
- Collaborate with researchers to design and identify repositories and issues that are challenging for LLMs.

Apply Now!

📌 Senior Software Engineer (Remote) (Canada)
🏢 Joblet-AI
📍 Canada