Content Evaluation Specialist (Remote)

Location: Remote within the United States, United Kingdom, Canada, Ireland, Australia & New Zealand (You must be legally authorised to work in the country where you are based)

Compensation: $15 - $18 per hour based on location, experience, responsibilities, and performance.

Role Overview

We are hiring on behalf of one of our clients, a leading global player in the AI space. This remote, part‑time contract role is focused on the detailed assessment and ranking of AI‑generated content. As a Specialist, you will apply rigorous grading rubrics to measure model performance, determining not just if a response is good, but how good it is compared to alternatives. Your evaluations will provide the essential data needed to fine‑tune the model’s accuracy, safety, and helpfulness.

Key Responsibilities

- Conduct in‑depth evaluations of AI responses across various topics, scoring them against specific metrics like factual accuracy, instruction following, and tone.
- Perform side‑by‑side (SxS) comparisons of model outputs, selecting the superior response and writing detailed justifications for your choice.
- Analyze content for specific failures, such as “hallucinations” (untrue facts), logical inconsistencies, or failure to meet negative constraints (e.g., “do not use lists”).




- Verify the safety of AI outputs by identifying and flagging bias, hate speech, or dangerous instructions according to strict safety guidelines.
- Collaborate with project leads to clarify ambiguities in the evaluation rubrics to ensure consistent grading across the team.

Qualifications

- Proven experience in content evaluation, data annotation, quality assurance, or editing.
- Robust critical thinking skills with the ability to objectively analyze text against a set of complex rules.
- Excellent reading comprehension and attention to detail to catch subtle nuances in user prompts and model responses.
- Strong research skills to quickly verify the accuracy of information provided by the AI.
- Comfortable working independently in a remote environment, managing time across multiple projects.

Compensation and Benefits

- Rate: Earn up to $18 USD/hr (Rates vary based on project complexity and expertise).
- Flexibility: 100% remote with no fixed schedule. Contributors typically spend 5–10 hours per week, with the option to work up to 40 hours.

Equal Opportunity Employer

We’re committed to fostering a diverse and inclusive work environment. We welcome applicants from all backgrounds and celebrate diversity in our workforce.

#J-18808-Ljbffr

📌 Content Evaluation Specialist (Remote) (Calgary)
🏢 Work Vista
📍 Calgary