This position is listed on behalf of a partner company, who manages all applications and next steps. We are looking for an experienced product leader to shape the foundations of an innovative AI platform designed to build, deploy, and manage intelligent agents at scale. In this role, you will own the agent runtime and evaluation infrastructure, enabling non-technical professionals to turn business needs into reliable, governed automations.

You will define the product vision and roadmap across agent lifecycles, model governance, testing, deployment, and monitoring. This is a hands-on leadership opportunity in an early-stage platform environment where experimentation, technical curiosity, and strong product judgment are essential. Your work will help transform traditional business operations into intelligent, agent-driven processes.

Own the vision, product strategy, and roadmap for the agent runtime platform, enabling non-technical users to create, test, publish, deploy, monitor, improve, and retire AI agents and automations without requiring traditional engineering support. Lead the development of agent lifecycle management capabilities, including version control, deployment processes, runtime management, observability, and continuous improvement. Define and evolve evaluation and model governance frameworks, including golden datasets, data labeling, automated evaluation, regression testing, and quality gates to ensure automations meet required standards before production.

Establish evaluation mechanisms for AI agents that assess other agents, ensuring quality, reliability, and measurable performance across large-scale automation environments. Lead the development of a model-agnostic platform, defining model routing, cost attribution, and pricing strategies that allow teams to select appropriate models for different use cases. Define integration strategies that enable agents to securely access internal systems, business data, and third-party applications through reusable connectors, credentials,



permissions, and access controls.

Develop oversight and monitoring experiences that allow business teams to manage large fleets of agents through dashboards, execution histories, traces, metrics, logs, audit trails, and emergency controls. Monitor emerging AI technologies, managed agent platforms, orchestration frameworks, and automation solutions to inform strategic build-versus-buy decisions and partnership opportunities. Provide product leadership across adjacent teams, guide complex technical and strategic decisions, mentor product colleagues, and establish high standards for product execution and customer experience. 7+ years of product management experience in B2B or enterprise SaaS environments, including at least 3 years working in AI-focused product roles. ~ Proven experience owning complex products end to end, from product vision and strategy through roadmap development, execution, launch, and continuous improvement. ~ Experience building and scaling AI agents or automation solutions that replace manual business processes, ideally in a zero-to-one product environment. ~ Strong practical understanding of AI evaluation methodologies, including golden datasets, data labeling, automated judges, regression testing, performance monitoring, and human-in-the-loop evaluation. ~ Ability to work closely with engineering teams on technical concepts such as agent-to-workflow contracts, structured outputs, deterministic execution, agent memory, and model orchestration. ~ Strong understanding of how non-technical users build, configure, deploy, and supervise automated systems,



with a focus on usability and operational reliability. ~ Hands‑on fluency with modern AI tools and large language models, including platforms such as Claude and Cursor, with the ability to independently prototype solutions and accelerate product development. ~ Ability to make informed decisions about AI models, evaluation frameworks, integrations, access controls, and deployment strategies. ~ Experience with managed agent platforms such as Claude Managed Agents or Workato Genies is a strong advantage, particularly regarding evaluations, connectors, permissions, versioning, and deployments. ~ Excellent written and verbal communication skills, with the ability to explain complex product decisions to engineers, designers, executives, and customers. ~ Demonstrated leadership and ability to influence multiple teams, establish shared product direction, and mentor or manage other product managers. ~ Strong product judgment, analytical thinking, customer empathy, and comfort navigating ambiguity in a rapidly evolving technology environment. ~ Ability to balance technical complexity with practical business outcomes, ensuring that platform capabilities translate into meaningful improvements for end users.

Equity: Opportunity to participate in the company's long-term growth through equity compensation. Medical, dental, and vision coverage for employees and their families.

Life insurance: Flexible time off: Flex Time Off in addition to company-paid holidays. Family leave, medical leave, and bereavement leave policies.

Retirement planning: Access to retirement savings plans.

Remote work support: Allowance to personalize your home office and technology setup.

Professional development: Annual stipend to support continued learning, training, and career growth.

Adaptable work environment: Remote-first collaboration with opportunities to work alongside talented colleagues across multiple locations. #

📌 Lead Product Manager, Agent Runtime & Evals (Winnipeg)
🏢 Jobgether
📍 Winnipeg