22 Nov
|
Cohere
|
Moncton
Red-team Large Language Models remotely for Cohere by creating adversarial prompts and testing unsafe behavior. This part-time contractor role is available across Canada and the United States.
As a Data Annotation Specialist on safety tasks, you will help prevent unintentional, adversarial, toxic, or unsafe outputs. Your work combines annotation, prompt engineering, model evaluation, and quality calibration across human-generated and model-generated tasks. You will operate independently within a globally distributed, asynchronous team while handling potentially explicit material.
Key Responsibilities:
- Develop prompts and scenarios that expose unsafe model responses
- Test evasive, over-refusing, and policy-inconsistent model behavior
- Label and rank outputs for safety and policy alignment
- Refine annotations and communicate recurring model failures
- Maintain accuracy across repetitive, high-volume assignments
Requirements:
- One or more years in AI annotation, LLM evaluation, or moderation
- Preferred exposure to quality assurance, red teaming, or prompt engineering
- Ability to apply nuanced guidelines to complex content
- Comfort evaluating sexual, violent, toxic, or disturbing material
- Solid detail orientation, written English, and remote time management
Combine red-teaming instincts, prompt creation, and careful annotation to strengthen Cohere’s Large Language Model safety.
📌 Large Language Model Red Team Annotator (Moncton)
🏢 Cohere
📍 Moncton