About The Role

Join Rex.zone’s AI Data Operations team to deliver high-quality labeled datasets that improve large language model evaluation and downstream model performance. You will label and review text, images, audio, and multimodal data while following strict annotation guidelines and quality targets.

Key Responsibilities

- Create accurate training labels for NLP and computer vision annotation tasks (text, image, audio, multimodal).
- Execute RLHF workflows (preference ranking, rationale writing, policy-aligned scoring) for LLM training pipelines.
- Perform QA evaluation, audit samples, and correct labeling errors to improve training data quality.
- Apply annotation guidelines compliance; document edge cases and reduce reviewer variance.
- Run prompt evaluation to score factuality, helpfulness, safety, and instruction adherence.
- Use taxonomies for content safety labeling (hate, harassment, self-harm, sexual content, violence) with consistent decision rules.
- Collaborate with project leads on rubrics, calibration sets, and acceptance criteria tied to model performance improvement.





Required Qualifications

- Mid-Senior experience in data labeling, data annotation, QA evaluation, or LLM evaluation.
- Robust written English for rubric-based judgments and rationale quality.
- Ability to follow annotation guidelines compliance and maintain consistency.
- Familiarity with NLP tasks such as named entity recognition and classification.
- Comfort working remotely in metrics-driven workflows.

Preferred Qualifications

- Exposure to RLHF and large language model evaluation methods.
- Experience with computer vision annotation (bounding boxes, polygons, segmentation).
- Experience labeling sensitive content with content safety policies.

Compensation

- Hourly rate: $30–$50 per hour
- Remote role across Canada

How To Apply Apply through Rex.zone and include relevant experience in data labeling, QA evaluation, and LLM/prompt evaluation. Share examples of guideline-driven work (rubrics, calibration, audits) if available.

📌 Data Labeling Jobs in Canada
🏢 Rex.zone
📍 Canada