Improve AI agent reliability at TribalScale with evaluation pipelines, quality metrics, and disciplined testing for agentic systems. The location remains to be confirmed. This Agentic QA Engineer role focuses on validating AI-driven workflows in real-world environments. You will design evaluation datasets, test scenarios, and benchmarks for dynamic application outputs. You will also collaborate with AI engineers, developers, product teams, and client-facing engineering teams. Key Responsibilities:
Develop pipelines measuring agent accuracy and task completion Establish quality metrics, datasets, scenarios, and benchmarks Evaluate edge cases, recovery, security,
and resilience Integrate quality checks into CI/CD pipelines Investigate production issues and improve test coverage Requirements:
Solid analytical thinking for complex real-world test cases Experience creating automated tests and testing frameworks Strong API, integration, regression, and end-to-end testing knowledge Experience debugging applications and technical issues Familiarity with modern software development practices Bring quality engineering expertise to AI agent evaluation, production reliability, and measurable software performance at TribalScale.
#J-18808-Ljbffr
📌 AI Agent Reliability Engineer (Winnipeg)
🏢 TribalScale
📍 Winnipeg
Reply to this offer
Impress this employer describing Your skills and abilities, fill out the form below and leave Your personal touch in the presentation letter.