- Role: Software Quality Engineer (Remote)
- Location: Remote (Work from Anywhere)
- Job Type: Full-Time
- Payout: Competitive, based on experience

Role Overview: We are hiring for one of our clients, seeking a Agentic Coding Annotator - Online / Offline Tasks to work on a full-time basis. This role involves working with realistic coding tasks in an agentic coding harness, reviewing model trajectories, verifying solutions, and producing high-quality annotations. Candidates are expected to read and debug code, validate behavior, follow detailed process rules, and make consistent judgment calls across model runs.

Key Responsibilities:

- Execute realistic coding tasks within the assigned agentic coding harness while maintaining model blindness and session independence
- Follow task instructions, milestones, planned interactions, and evaluation guardrails consistently across runs
- Verify model outputs by reading code, running commands, checking logs, and inspecting generated artifacts
- Design multi-step, realistic coding tasks including user intent and milestone structure for offline evaluations
- Write explicit, specific, evidence-based rationales for trajectory rankings and assessments





Required Skills & Qualifications:

- Proficiency in Python, JavaScript, Java, C++, or SQL for code analysis and debugging
- Experience working with realistic software tasks beyond toy problems or shallow code-review exercises
- Ability to design task-specific rubrics and binary evaluation criteria for model outputs
- Strong attention to detail to ensure high-quality annotations and consistent grading
- Comfort with debugging, test validation, and manual verification of software behavior

More About the Prospect:

This role offers a unique opportunity to work with a global leader in the Technology, Information and Internet industry, contributing to the advancement and deployment of powerful AI systems. The work directly supports the development of agentic coding models used for mission-critical applications.

Equal Opportunity Employer:

We hire based on skills and expertise. All qualified candidates are welcome regardless of background, experience, or prior employment history. Applications are reviewed solely on demonstrated technical ability and qualifications.

Apply Now!

📌 Software Quality Engineer (Remote) (Canada)
🏢 Jobs Ai
📍 Canada