Position: Shopify Shop App Agent Annotator

Type: Contract

Compensation: $40–$50/hour

Location: Remote

Commitment: ~5 hours/week

Role Responsibilities

- Evaluate full shopper-agent conversations, including tool calls and their outcomes.
- Identify the specific turn that received a thumbs-down from the shopper.
- Classify the issue with a primary domain tag and a specific secondary tag.
- Write concise comments explaining the shopper's intent, the agent's actions, and the rationale for the tags.
- Work independently and asynchronously to maintain consistency in tagging across conversations.

Qualifications

Must-Have

- Rule discipline to apply a fixed taxonomy consistently.
- Comfort reading structured traces to understand agent actions.
- Restraint under ambiguity to use the unknown label appropriately.

Preferred

- Consumer e-commerce fluency in orders, tracking,



and platform responsibilities.
- Prior annotation, labelling, or model-evaluation work with a defined rubric.
- Experience with the Shop app or other AI shopping assistants.
- Familiarity with customer-support operations and escalation paths.

Application Process

- Upload resume
- AI interview based on your resume

Resources & Support

- For details about the interview process and platform information, please check: https://talent.docs.mercor.com/welcome
- For any help or support, reach out to: [email protected]

PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this chance.

📌 Marchandiseur/marchandiseuse - commerce de détail (Calgary)
🏢 Neptune Retail Solutions
📍 Calgary