Location:
Remote Commitment: Part-time (20-40 hours/week) Role Responsibilities
- Build and maintain pipelines and infrastructure supporting AI data generation, collection, and evaluation workflows.
- Design, refine, and iterate on rubrics and evaluation criteria for preference data.
- Review and label preference data while identifying subtle quality issues.
- Collaborate with researchers and engineers to translate qualitative judgment into scalable evaluation processes.
- Synthesize technical findings into transparent, actionable write-ups, updates, and recommendations. Requirements
- 5+ years of experience with Python, TypeScript, Golang, or JavaScript,
particularly building and maintaining pipelines or infrastructure.
- Hands-on experience with preference data collection and/or rubric creation.
- Experience navigating or contributing to large codebases; experience with codebases such as VSCode is a plus.
- Experience with AI/LLM evaluation or related engineering workflows is preferred.
- A LinkedIn profile is mandatory for review.
- Candidates should not currently be working with Microsoft. Application Process
- Apply via the Easy Apply.
- Check your email for next steps.