- Own agents, the agent builder, the evaluation builder, and agentic memory as products, including their roadmaps, sequencing, and business outcomes

- Own the applied-science delivery roadmap and cross-team orchestration for the Caseware AI platform

- Sequence evaluation, agentic-memory, and platform workstreams into a single delivery plan

- Own the product acceptance bar for every AI feature and establish evaluation baselines

- Work with domain experts and partner accounting and audit firms to validate AI outputs against professional standards

- Serve as the practitioner voice and SME proxy by shadowing audit workflows, testing prototypes, and identifying actionable failure modes

- Partner daily with engineering to convert problem statements into requirements and acceptance criteria

- Monitor AI model developments, architectures, agent patterns, and competitor activity

- Define and own product health metrics, including task completion, correction rates, override frequency, trust signals, and time-to-completion

- Collaborate with applied scientists, engineers, domain experts, and external partner firms

- Make autonomous product decisions and own product outcomes

Requirements

- 6+ years of experience in technology-focused product management or product development

- 3–4 years working deeply with AI or machine learning–driven products

- Software engineers transitioning into product management are welcome

- Proven ability to drive delivery across multiple technical teams, coordinating dependencies, shared interfaces, and timelines across applied science and platform engineering

- Direct experience building and running evaluation frameworks for production AI products

- Experience defining acceptance bars,



running human-in-the-loop and practitioner validation, tracking regressions across releases, and using results to decide what ships

- Technical fluency in LLMs, RAG, prompt engineering, embeddings, and agentic patterns

- High agency and ownership of outcomes

- Intellectual curiosity about AI, including following model releases, reading research, and understanding frontier-lab developments

- Experience representing the voice of customers or practitioners and working with domain experts or subject matter specialists

- Experience defining quality bars in high-stakes, regulated, or qualified-grade contexts

- Comfort with ambiguity

- Genuine curiosity about auditing and appreciation for high-quality work

- Willingness to build accounting and audit domain fluency on the job

Core Competencies

Demonstrates expertise in technology-focused product management with a strong emphasis on AI and machine learning-driven products. Capable of defining product health metrics, managing cross-team orchestration, and ensuring high-quality outcomes in regulated environments.

Highest-signal resume keywords

- Product Management

- AI Product Development

- Evaluation Frameworks

- Technical Fluency in LLMs

- Cross-Team Collaboration

Hard Skills

- Product Roadmap Ownership

- AI Feature Evaluation

- Acceptance Criteria Definition

- Human-in-the-Loop Validation

- Prompt Engineering

- Agentic Patterns

- Task Completion Metrics

- Correction Rates

- Override Frequency

- Time-to-Completion

Soft Skills

- High Agency

- Intellectual Curiosity

- Comfort with Ambiguity

Industry Keywords

- Applied Science

- Audit Workflows

- Professional Standards

- Domain Expertise

- Accounting Fluency

📌 Technical Product Manager – AI Products (Toronto)
🏢 Jobtailor
📍 Toronto