Overview

Rex.zone is hiring for Senior Data Labeling roles supporting AI/ML training workflows end to end, including data labeling, RLHF, prompt evaluation, QA evaluation, and training data quality audits for large language model evaluation and computer vision annotation. This is a full time, remote prospect aligned with Ottawa talent.

What You Will Do
Execute senior-level data labeling across text, image, and multimodal tasks with consistent decision-making and high accuracy
Perform RLHF-style preference labeling/ranking using rubric-based guidelines
Run prompt evaluation and QA evaluation to identify failure modes and policy gaps
Audit training data quality via sampling, error categorization, and root-cause analysis
Ensure annotation guidelines compliance; propose clarifications and edge-case examples
Support NER/span labeling and content safety labeling as needed
Resolve disagreements, calibrate annotators,



and stabilize inter-annotator agreement
Document decisions and acceptance criteria for downstream model training teams

Core Workflows
Gold set creation, consensus labeling, disagreement analysis
Quality systems: spot checks, batch acceptance, error tracking, corrective actions
Evaluation datasets for LLMs (rubric scoring, adversarial/safety probes)
Computer vision annotation (bounding boxes, polygons, segmentation) where applicable

Who You Are
Experienced in data labeling/annotation with solid edge-case handling and consistency
Comfortable with RLHF, prompt evaluation, and QA evaluation concepts
Able to mentor peers and communicate clearly when guidelines conflict with real data

📌 Senior Data Labeling Jobs In Ottawa Remote (Canada)
🏢 Rex.zone
📍 Canada