Remote Data Labeling Specialist (Vancouver) — Full-Time

Rex.zone is hiring a Remote Data Labeling Specialist to deliver consistent, policy-aligned annotations used to train and evaluate AI/ML models. You will work across NLP labeling, RLHF preference ranking, prompt evaluation, computer vision annotation, and content safety labeling to improve training data quality and downstream model performance.

Key Responsibilities

- Apply annotation guidelines accurately across NLP and computer vision tasks

- Perform RLHF preference ranking and rubric-based evaluations to align model behavior with human preferences

- Execute prompt evaluation and response quality scoring for LLM evaluation workflows

- Conduct QA evaluation, sampling audits, guideline compliance checks, and error categorization to reduce label noise

- Label content safety categories (e.g., harassment, self-harm, hate) using policy taxonomies

- Collaborate with project leads to resolve edge cases, refine rubrics, and document decisions

- Track productivity and accuracy metrics to support model performance improvement

Workstreams You May Support

- NLP data labeling: named entity recognition (NER), intent classification, sentiment/topic labeling, summarization evaluation

- LLM training pipelines: instruction-following evaluation, pairwise preference ranking, rubric grading

- Computer vision annotation: bounding boxes, polygons,



keypoints, segmentation, OCR validation

- Content safety labeling: policy-based moderation labels and severity scoring

- Multimodal tasks: text-image relevance, caption validation, VQA-style evaluation

Required Qualifications

- Experience applying structured guidelines to data annotation and/or QA evaluation work

- Strong written communication for consistent labeling rationale and disagreement resolution

- Comfort working with ambiguity, edge cases, and evolving rubrics

- Familiarity with NLP, RLHF, or LLM evaluation concepts (training data quality, preference ranking, prompt evaluation)

- Ability to maintain accuracy at production scale and meet full time delivery expectations

Preferred Qualifications

- Experience with annotation platforms and audit workflows

- Exposure to computer vision annotation tools (boxes, polygons, segmentation)

- Experience with content safety labeling or policy taxonomy interpretation

- Experience documenting guidelines, building QA checklists, or running inter-annotator agreement checks

Remote Work Notes

This is a Remote role. You can be based in Vancouver while working with distributed teams. Projects may require overlap with US business hours, and task availability can vary by domain.

Pay: $30–$50 per hour (base salary).

📌 Remote Data Labeling Specialist (Vancouver)
🏢 Rex.zone
📍 Vancouver