Join IDC as an AI Engineer focused on AI Observability and Quality, where you'll develop automated evaluation systems for our AI platform. You'll ensure high-quality outputs and enhance system integrity through rigorous testing.
As the first hire in this new function, you will design and build the evaluation infrastructure for IDC's next-generation AI-powered intelligence platform. This solo role requires a robust background in testing frameworks and machine learning evaluation, enabling you to create automated test suites and regression detection systems. Your expertise will play a critical role in maintaining the platform's credibility and driving high standards.
Key Responsibilities:
• Design evaluation infrastructure for AI systems
• Build automated test suites for answer validation




• Develop regression detection systems to prevent issues
• Create evaluation frameworks measuring response accuracy
• Track operational efficiency of AI pipeline execution
Requirements:
• 6+ years in software engineering, with focus on testing
• Proven experience with ML evaluation frameworks
• Proficiency in Python and independent operation
• Knowledge of generative AI quality measurement
• Background in A/B testing and information retrieval metrics
Your contributions will directly impact IDC's mission to shape technology understanding and decision-making.
#J-18808-Ljbffr

📌 AI Engineer for Quality at IDC (Toronto)
🏢 Socket.dev
📍 Toronto