- Own agents, the agent builder, the evaluation builder, and agentic memory as products, including their roadmaps, sequencing, and business outcomes - Own the applied-science delivery roadmap and cross-team orchestration for the Caseware AI platform - Sequence evaluation, agentic-memory, and platform workstreams into a single delivery plan - Own the product acceptance bar for every AI feature and establish evaluation baselines - Work with domain experts and partner accounting and audit firms to validate AI outputs against professional standards - Serve as the practitioner voice and SME proxy by shadowing audit workflows, testing prototypes, and identifying actionable failure modes - Partner daily with engineering to convert problem statements into requirements and acceptance criteria - Monitor AI model developments, architectures, agent patterns, and competitor activity - Define and own product health metrics, including task completion, correction rates, override frequency, trust signals, and time-to-completion - Collaborate with applied scientists, engineers, domain experts, and external partner firms - Make autonomous product decisions and own product outcomes Requirements - 6+ years of experience in technology-focused product management or product development - 3–4 years working deeply with AI or machine learning–driven products - Software engineers transitioning into product management are welcome - Proven ability to drive delivery across multiple technical teams, coordinating dependencies, shared interfaces, and timelines across applied science and platform engineering - Direct experience building and running evaluation frameworks for production AI products - Experience defining acceptance bars,



running human-in-the-loop and practitioner validation, tracking regressions across releases, and using results to decide what ships - Technical fluency in LLMs, RAG, prompt engineering, embeddings, and agentic patterns - High agency and ownership of outcomes - Intellectual curiosity about AI, including following model releases, reading research, and understanding frontier-lab developments - Experience representing the voice of customers or practitioners and working with domain experts or subject matter specialists - Experience defining quality bars in high-stakes, regulated, or professional-grade contexts - Comfort with ambiguity - Genuine curiosity about auditing and appreciation for high-quality work - Willingness to build accounting and audit domain fluency on the job Core Competencies Demonstrates expertise in technology-focused product management with a strong emphasis on AI and machine learning-driven products. Capable of defining product health metrics, managing cross-team orchestration, and ensuring high-quality outcomes in regulated environments. Highest-signal resume keywords - Product Management - AI Product Development - Evaluation Frameworks - Technical Fluency in LLMs - Cross-Team Collaboration Hard Skills - Product Roadmap Ownership - AI Feature Evaluation - Acceptance Criteria Definition - Human-in-the-Loop Validation - Prompt Engineering - Agentic Patterns - Task Completion Metrics - Correction Rates - Override Frequency - Time-to-Completion Soft Skills - High Agency - Intellectual Curiosity - Comfort with Ambiguity Industry Keywords - Applied Science - Audit Workflows - Skilled Standards - Domain Expertise - Accounting Fluency

📌 Technical Product Manager – AI Products (Toronto)
🏢 Jobtailor
📍 Toronto