About The Role

Support AI/ML training pipelines by performing remote data annotation and evaluation across NLP, computer vision, and content safety. You will label and review datasets, complete RLHF preference ranking, and deliver rubric-based prompt and QA evaluations to improve training data quality and model performance.

What You Will Do

- Execute data labeling for text, image, and multimodal tasks using web-based tools

- Perform named entity recognition (NER), text classification, and span labeling as required

- Complete RLHF pairwise comparisons and preference ranking with clear rationale notes

- Run prompt evaluation using rubrics for helpfulness, correctness, and safety

- Conduct QA evaluation, sampling, and audits to ensure guideline compliance and consistency

- Annotate computer vision data (bounding boxes, polygons, segmentation masks) when assigned

- Perform content safety labeling for sensitive or policy-violating material when needed

- Document edge cases, track disagreements, and propose guideline clarifications

- Collaborate asynchronously with distributed teams to meet throughput and quality targets

Required Qualifications

- Experience with professional data annotation, data labeling, or evaluation workflows





- Strong written English for rubric-based judgments and structured feedback

- Ability to follow detailed annotation guidelines and maintain consistency under QA review

- Familiarity with NLP concepts (classification, NER) and/or computer vision annotation fundamentals

- Comfort working with online labeling tools, spreadsheets, and documentation

- Reliable remote work setup and availability for full time schedules

Preferred Qualifications

- Hands-on experience with RLHF evaluation, prompt evaluation, or LLM output grading

- Experience conducting QA evaluation, calibration sessions, or inter-annotator agreement checks

- Understanding of LLM failure modes (hallucinations, prompt injection, unsafe outputs)

- Exposure to taxonomy/ontology or guideline authoring

Compensation

Hourly pay range: $30–$50 USD (HOURLY), based on task complexity, performance, and project needs.

How To Apply

Apply via Rex.zone and include your annotation experience plus examples of RLHF, prompt evaluation, and QA evaluation work that demonstrates strong guideline adherence and training data quality focus.

📌 Remote Data Annotation Jobs in Vancouver
🏢 Rex.zone
📍 Vancouver