Design generative AI evaluation systems for EA-SPORTS FC at Electronic Arts, combining LLM-as-judge patterns, metrics, and automated testing. Work hybrid in Burnaby, BC.
This Software Engineer III role sits within FC Quality Verification Engineering. You will create evaluation frameworks and developer tools that identify quality issues earlier. Your work spans dataset curation, model evaluation, validation criteria, error analysis, and deterministic checks.
Key Responsibilities: Develop evaluation frameworks for generative AI systems
Implement LLM-as-judge patterns and rubric-based scoring
Perform automated calibration alongside deterministic validation checks
Define methods for exploring complex software behaviour




Produce reproducible diagnostics and support delivered software
Requirements: Experience with generative system evaluation frameworks
Knowledge of LLM architectures, concepts, and failure modes
Experience curating datasets and defining validation criteria
Experience with quantitative and qualitative evaluation metrics
Master of Science or equivalent engineering experience
Bring generative AI evaluation, LLM analysis, and software testing expertise to Electronic Arts’ EA-SPORTS FC engineering work.

#J-18808-Ljbffr

📌 Generative AI Evaluation Engineer (British Columbia)
🏢 Electronic Arts (EA)
📍 British Columbia