Design generative AI evaluation systems for EA-SPORTS FC at Electronic Arts, combining LLM-as-judge patterns, metrics, and automated testing. Work hybrid in Burnaby, BC. This Software Engineer III role sits within FC Quality Verification Engineering. You will create evaluation frameworks and developer tools that identify quality issues earlier. Your work spans dataset curation, model evaluation, validation criteria, error analysis, and deterministic checks. Key Responsibilities:





Develop evaluation frameworks for generative AI systems Implement LLM-as-judge patterns and rubric-based scoring Perform automated calibration alongside deterministic validation checks Define methods for exploring complex software behaviour Produce reproducible diagnostics and support delivered software Requirements:

Experience with generative system evaluation frameworks Knowledge of LLM architectures, concepts, and failure modes Experience curating datasets and defining validation criteria Experience with quantitative and qualitative evaluation metrics Master of Science or equivalent engineering experience Bring generative AI evaluation, LLM analysis, and software testing expertise to Electronic Arts’ EA-SPORTS FC engineering work.

#J-18808-Ljbffr

📌 Generative AI Evaluation Engineer (Winnipeg)
🏢 Electronic Arts (EA)
📍 Winnipeg