Video Evaluation Generalist

Position: Video Evaluation Generalist

Type: Hourly Contract

Compensation: $15–$20/hour

Location: Remote

About the Opportunity

Mercor is seeking detail-oriented writing and critical-thinking professionals to contribute to a high-impact video AI research project with a leading AI lab.

You will evaluate short AI-generated video clips by comparing paired videos against structured evaluation rubrics. Your assessments will focus on prompt adherence, motion quality, visual artifacts, physical plausibility, and overall output quality. This is a flexible, short-term opportunity for professionals who can make consistent judgments and communicate their reasoning clearly.

Responsibilities

- Watch and evaluate paired short video clips presented side by side.
- Compare video outputs against provided evaluation rubrics.
- Assess dimensions including prompt adherence, motion quality, visual artifacts, and physical plausibility.
- Assign appropriate rubric scores to each video.
- Select an overall preference between paired video outputs based on the evaluation criteria.
- Provide concise, clear written justifications for evaluation decisions when required.
- Apply consistent standards across high volumes of evaluation tasks.
- Maintain strong calibration with the provided rubric throughout the project.
- Identify meaningful differences in video quality and communicate judgments accurately.
- Complete assigned tasks independently while meeting project deadlines.

Required Qualifications

- BA or BS degree from a reputable institution, either completed or currently in progress.
- Strong writing and critical-thinking skills.
- Excellent attention to detail.




- Ability to carefully evaluate visual content against structured criteria.
- Strong ability to make consistent comparative judgments.
- Ability to clearly explain preferences and scoring decisions in writing.
- Ability to work independently and meet deadlines.
- Comfortable working in a structured, high-volume evaluation environment.

Preferred Qualifications

- 2+ years of experience in teaching or research.
- Experience with content evaluation, annotation, quality assurance, or academic assessment.
- Experience interpreting and applying detailed rubrics or evaluation frameworks.
- Strong instructional writing or technical communication skills.
- Previous exposure to AI-generated content or AI evaluation projects is a plus.

Compensation

- $15–$20/hour
- Hourly contract engagement.
- Fully remote and flexible.
- Expected commitment of at least 20 hours per week.
- Short-term engagement of approximately 1.5 weeks.
- Opportunity to contribute to a high-impact video AI research project.

Eligibility

- Applicants with a BA or BS degree from a reputable institution, completed or in progress.
- Solid writing and critical-thinking abilities are required.
- Candidates able to commit at least 20 hours per week for approximately 1.5 weeks are encouraged to apply.

Application Process

1. Apply for the Video Evaluation Generalist position.
2. Complete the AI-led interview, which takes approximately 15 minutes.
3. Receive an invitation to join the project if selected.
4. Complete the required onboarding and project instructions.
5. Begin evaluating paired video outputs against the provided rubrics.
6. Maintain consistent scoring and complete assigned tasks throughout the project.

📌 Video Content Creation Expert (Remote | ) (Canada)
🏢 Synthires
📍 Canada