Join IDC as an AI Engineer focused on AI Observability and Quality, where you'll develop automated evaluation systems for our AI platform. You'll ensure high-quality outputs and enhance system integrity through rigorous testing. As the first hire in this new function, you will design and build the evaluation infrastructure for IDC's next-generation AI-powered intelligence platform.

This solo role requires a robust background in testing frameworks and machine learning evaluation, enabling you to create automated test suites and regression detection systems. Your expertise will play a critical role in maintaining the platform's credibility and driving high standards. Key Responsibilities:

- Design evaluation infrastructure for AI systems
- Build automated test suites for answer validation




- Develop regression detection systems to prevent issues
- Create evaluation frameworks measuring response accuracy
- Track operational efficiency of AI pipeline execution Requirements:
- 6+ years in software engineering, with focus on testing
- Proven experience with ML evaluation frameworks
- Proficiency in Python and independent operation
- Knowledge of generative AI quality measurement
- Background in A/B testing and information retrieval metrics Your contributions will directly impact IDC's mission to shape technology understanding and decision-making.

📌 AI Engineer for Quality at IDC (Toronto)
🏢 Socket.dev
📍 Toronto