13 Nov
|
TribalScale
|
Winnipeg
13 Nov
TribalScale
Winnipeg
Improve AI agent reliability at TribalScale with evaluation pipelines, quality metrics, and disciplined testing for agentic systems. The location remains to be confirmed. This Agentic QA Engineer role focuses on validating AI-driven workflows in real-world environments. You will design evaluation datasets, test scenarios, and benchmarks for dynamic application outputs. You will also collaborate with AI engineers, developers, product teams, and client-facing engineering teams. Key Responsibilities:
Develop pipelines measuring agent accuracy and task completion Establish quality metrics, datasets, scenarios, and benchmarks Evaluate edge cases, recovery, security, and resilience Integrate quality checks into CI/CD pipelines Investigate production issues and improve test coverage Requirements:
Strong analytical thinking for complex real-world test cases Experience creating automated tests and testing frameworks Strong API, integration, regression, and end-to-end testing knowledge Experience debugging applications and technical issues Familiarity with up-to-date software development practices Bring quality engineering expertise to AI agent evaluation, production reliability, and measurable software performance at TribalScale.
📌 AI Agent Reliability Engineer (Winnipeg)
🏢 TribalScale
📍 Winnipeg