- Own agents, the agent builder, the evaluation builder, and agentic memory as products, including their roadmaps, sequencing, and business outcomes
- Own the applied-science delivery roadmap and cross-team orchestration for the Caseware AI platform
- Sequence evaluation, agentic-memory, and platform workstreams into a single delivery plan
- Own the product acceptance bar for every AI feature and establish evaluation baselines
- Work with domain experts and partner accounting and audit firms to validate AI outputs against skilled standards
- Serve as the practitioner voice and SME proxy by shadowing audit workflows, testing prototypes, and identifying actionable failure modes
- Partner daily with engineering to convert problem statements into requirements and acceptance criteria
- Monitor AI model developments, architectures, agent patterns, and competitor activity
- Define and own product health metrics, including task completion, correction rates, override frequency, trust signals, and time-to-completion
- Collaborate with applied scientists, engineers, domain experts, and external partner firms
- Make autonomous product decisions and own product outcomes

Requirements

- 6+ years of experience in technology-focused product management or product development
- 3–4 years working deeply with AI or machine learning–driven products
- Software engineers transitioning into product management are welcome
- Proven ability to drive delivery across multiple technical teams, coordinating dependencies, shared interfaces, and timelines across applied science and platform engineering
- Direct experience building and running evaluation frameworks for production AI products
- Experience defining acceptance bars,



running human-in-the-loop and practitioner validation, tracking regressions across releases, and using results to decide what ships
- Technical fluency in LLMs, RAG, prompt engineering, embeddings, and agentic patterns
- High agency and ownership of outcomes
- Intellectual curiosity about AI, including following model releases, reading research, and understanding frontier-lab developments
- Experience representing the voice of customers or practitioners and working with domain experts or subject matter specialists
- Experience defining quality bars in high-stakes, regulated, or professional-grade contexts
- Comfort with ambiguity
- Genuine curiosity about auditing and appreciation for high-quality work
- Willingness to build accounting and audit domain fluency on the job

Core Competencies

Demonstrates expertise in technology-focused product management with a strong emphasis on AI and machine learning-driven products. Capable of defining product health metrics, managing cross-team orchestration, and ensuring high-quality outcomes in regulated environments.

Highest-signal resume keywords

- Product Management
- AI Product Development
- Evaluation Frameworks
- Technical Fluency in LLMs
- Cross-Team Collaboration

Hard Skills

- Product Roadmap Ownership
- AI Feature Evaluation
- Acceptance Criteria Definition
- Human-in-the-Loop Validation
- Prompt Engineering
- Agentic Patterns
- Task Completion Metrics
- Correction Rates
- Override Frequency
- Time-to-Completion

Soft Skills

- High Agency
- Intellectual Curiosity
- Comfort with Ambiguity

Industry Keywords

- Applied Science
- Audit Workflows
- Professional Standards
- Domain Expertise
- Accounting Fluency

#J-18808-Ljbffr

📌 Technical Product Manager – AI Products (Toronto)
🏢 Jobtailor
📍 Toronto