Machine Learning Evaluation Engineers shape quality for Apple Creativity Apps using model evaluation, datasets, and scalable infrastructure. The role focuses on AI features from development through shipment.Apple’s Creativity Apps team seeks an engineer with five or more years in machine learning or software engineering. You will define evaluation methodologies, metrics, and testing strategies for models, large language models, and AI features.

Your work connects data curation, infrastructure, quality reporting, and cross-functional collaboration across model development and customer-facing features.Key Responsibilities:

Define evaluation strategies and metrics for models, LLMs,



and featuresCreate representative datasets and improve their quality iterativelyBuild scalable infrastructure with reproducible runs and tracked baselinesApply automated judging, test generation, and agentic workflowsDrive quality issues from model checkpoints through integrated featuresRequirements:

Five or more years in machine learning or software engineeringExperience evaluating ML models, LLMs, and AI productsUnderstanding of data, training, deployment, and monitoring cyclesProficiency in Python, Swift, or comparable languagesBachelor’s degree or equivalent experience in a related fieldApply machine learning, LLM, dataset, and quality expertise to evaluate Apple creativity features at scale.

📌 Machine Learning Evaluation Engineer (Winnipeg)
🏢 Apple
📍 Winnipeg