About The Role What if your years of hard-earned research expertise could directly shape the future of AI? We're looking for domain experts with deep machine learning knowledge to design evaluation challenges that push state-of-the-art AI systems to their limits — the kind of problems only a true specialist could craft.About The Role What if your years of hard-earned research expertise could directly shape the future of AI? We're looking for domain experts with deep machine learning knowledge to design evaluation challenges that push state-of-the-art AI systems to their limits — the kind of problems only a true specialist could craft.Your work won't sit in a drawer. It directly influences how the next generation of AI models are measured, trained, and improved.Organization: AlignerrType: Hourly ContractLocation: RemoteCommitment: 10–40 hours/weekWhat You'll Do Design complex, original machine learning problems rooted in your area of domain expertiseCreate evaluation tasks that demand advanced knowledge well beyond standard ML pipelinesDraw from your own research experience to craft challenges that genuinely test highly capable AI modelsWrite explicit problem statements, define evaluation criteria, and establish gold-standard solutionsAssess AI-generated solutions for correctness, creativity, and methodological rigorDocument problem difficulty, required domain knowledge, and expected failure modesCollaborate asynchronously with a global team of researchers and engineersWho You Are Graduate-level expertise (MS or Ph D preferred) in a scientific or technical discipline that intersects with machine learningStrong working knowledge of ML methods — model selection,



feature engineering, evaluation metrics, and pipeline designDeep familiarity with active, open research problems in your fieldA sharp eye for where general ML knowledge breaks down and specialized domain insight becomes essentialExperience publishing or conducting original research is highly valuedExcellent written communication — you can articulate complex, nuanced problems with precision and claritySelf-motivated and energized by intellectually demanding, independent workExample Domains We welcome experts from a wide range of fields, including but not limited to:Computational biology, genomics, or bioinformaticsClimate science and environmental modelingMedical imaging and healthcare MLMaterials science and computational chemistryAstrophysics and signal processingNatural language processing for low-resource or specialized corporaRobotics, control theory, or reinforcement learning in complex environmentsFinancial modeling and quantitative analysisIf your domain sits at the frontier of ML research, we want to hear from you.Why Join Us Work at the cutting edge — your challenges help define the boundaries of what AI can and cannot doMake a real impact — your expertise directly shapes AI safety and evaluation researchFull autonomy — work on your own schedule, from anywhere in the worldFlexible commitment — scale hours up or down based on your availabilityOngoing opportunity — strong contributors are considered for contract extensions and deeper research involvementBuild your profile — establish yourself as a contributor to frontier AI development alongside top research labs#J-18808-Ljbffr

📌 Machine Learning Evaluation Specialist (Toronto)
🏢 Alignerr
📍 Toronto