Develop generative AI evaluation frameworks for EA SPORTS FC from Electronic Arts’ Burnaby, BC office, working hybrid at least three days weekly.

As a Software Engineer III, you will create tools for testing AI and machine learning systems. Your work will cover LLM-as-a-judge patterns, rubric-based scoring, calibration, datasets, and error analysis. You will collaborate with engineering, design, and quality partners to clarify requirements and resolve blockers.

Key Responsibilities:

- Develop evaluation frameworks for generative AI systems
- Implement LLM-as-a-judge patterns and rubric-based scoring
- Curate datasets and define validation criteria
- Perform quantitative and qualitative error analysis
- Collaborate with engineering, design, and quality partners

Requirements:

- Hands-on experience evaluating AI/ML and LLM-based systems
- Experience with generative system evaluation frameworks
- Knowledge of prompting, fine-tuning, or retrieval-augmented generation
- Experience using quantitative machine learning metrics
- 5+ years of skilled software engineering experience

Bring generative AI, NLP, evaluation framework, and LLM expertise to quality engineering at Electronic Arts.

📌 Generative AI Evaluation Engineer EA (Vancouver)
🏢 Electronic Arts
📍 Vancouver