Health AI Conversation Evaluator (Canada)

Health AI Conversation Evaluator (Canada)

18 Nov
|
Default Project
|
Canada

18 Nov

Default Project

Canada

Job DescriptionJob Description

About the job

n

Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark, General Catalyst, Peter Thiel, Adam D'Angelo, Larry Summers, and Jack Dorsey.

n

Position: Generalist Annotator — Health AI Conversation Quality Evaluation
nType: Contract
nCompensation: $20–$160/hour
nLocation: Remote

n

Role Responsibilities

n

nEvaluate the relevance and clarity of AI responses. Ensure the AI answers the question in understandable language for non-experts.nDetect sycophancy in AI interactions. Identify when the AI tells users what they want to hear instead of what they need.nAssess the tone and outcome of conversations. Determine if the user leaves informed and calm or anxious and uninformed.nProvide written justification for ratings below the top option. Explain what went wrong and suggest improvements.nWork independently to review conversations ranging from 1 to 7 turns. Spend 2–13 minutes per conversation based on length.nn

Qualifications

n

Must-Have

n





nStrong written English skills.nAbility to explain judgments in two to four specific sentences.nCareful reading to notice omissions in responses.nComfort with applying a detailed rubric consistently.nReliable availability for time-boxed batches.nn

Preferred

n

nExperience in annotation, human-feedback, evaluation, or content review.nBackground in explaining technical concepts to non-experts.nExperience with rubric-based grading or quality assurance.nn

Important

n

nUsing an AI tool to write comments is prohibited. This work captures human judgment that AI lacks. AI-written feedback corrupts the dataset and is checked for.nn

Application Process (Takes 20–30 mins to complete)

n

nUpload resumenAI interview based on your resumenSubmit formnn

Resources & Support

n

nFor details about the interview process and platform information, please check: https://talent.docs.mercor.com/welcomenFor any help or support, reach out to: [email protected]

PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this prospect.

📌 Health AI Conversation Evaluator (Canada)
🏢 Default Project
📍 Canada

Reply to this offer

Impress this employer describing Your skills and abilities, fill out the form below and leave Your personal touch in the presentation letter.

Subscribe to this job alert:

Get the latest job offers by email for: health ai conversation evaluator (canada) / canada

Subscribe to this job alert:

Get the latest job offers by email for: health ai conversation evaluator (canada) / canada