About The Role

Support AI/ML training pipelines by performing remote data annotation and evaluation across NLP, computer vision, and content safety. You will label and review datasets, complete RLHF preference ranking, and deliver rubric-based prompt and QA evaluations to improve training data quality and model performance.

What You Will Do
Execute data labeling for text, image, and multimodal tasks using web-based tools
Perform named entity recognition (NER), text classification, and span labeling as required
Complete RLHF pairwise comparisons and preference ranking with clear rationale notes
Run prompt evaluation using rubrics for helpfulness, correctness, and safety
Conduct QA evaluation, sampling, and audits to ensure guideline compliance and consistency
Annotate computer vision data (bounding boxes, polygons, segmentation masks) when assigned
Perform content safety labeling for sensitive or policy-violating material when needed
Document edge cases, track disagreements, and propose guideline clarifications
Collaborate asynchronously with distributed teams to meet throughput and quality targets

Required Qualifications
Experience with qualified data annotation, data labeling, or evaluation workflows




Robust written English for rubric-based judgments and structured feedback
Ability to follow detailed annotation guidelines and maintain consistency under QA review
Familiarity with NLP concepts (classification, NER) and/or computer vision annotation fundamentals
Comfort working with online labeling tools, spreadsheets, and documentation
Reliable remote work setup and availability for full time schedules

Preferred Qualifications
Hands-on experience with RLHF evaluation, prompt evaluation, or LLM output grading
Experience conducting QA evaluation, calibration sessions, or inter-annotator agreement checks
Understanding of LLM failure modes (hallucinations, prompt injection, unsafe outputs)
Exposure to taxonomy/ontology or guideline authoring

Compensation

Hourly pay range: $30–$50 USD (HOURLY), based on task complexity, performance, and project needs.

How To Apply

Apply via Rex.zone and include your annotation experience plus examples of RLHF, prompt evaluation, and QA evaluation work that demonstrates strong guideline adherence and training data quality focus.

📌 Remote Data Annotation Jobs In Vancouver
🏢 Rex.zone
📍 Vancouver