Build automated quality systems for Poka’s AI assistant using Python, TypeScript, and evaluation tools. This is a hybrid or remote position based on location.

As a quality assurance specialist, you will test assistant answers and surrounding features without manual regression passes. You will define quality with development, product, and design teams while building evaluations, automated tests, and practical tools. Your work supports the connected worker platform used across manufacturing sites by global leaders.

Key Responsibilities:
Automate QA for features and build tools for evaluation analysis
Maintain evaluation suites using real questions and production edge cases
Write scoring rubrics and calibrate LLM judges against human samples
Run CI regression checks for quality,



cost, and speed
Review conversations, identify failure patterns, and support fixes

Requirements:
Two to four years of relevant experience
AEC, DEC, BAC, or equivalent experience
Background in test automation or software development engineering testing
Comfort with Python, TypeScript, and several programming languages
Curiosity about LLM behavior and manufacturing users’ needs

Apply quality automation, AI evaluation, and programming skills to improve Poka’s manufacturing assistant for real users.
J-18808-Ljbffr

📌 Ai Assistant Quality Automation Developer Montreal
🏢 Poka
📍 Montreal