Role overview Cohere builds agentic AI infrastructure for enterprises, and this role develops the platform enabling secure code execution, model routing, and resource management for autonomous AI agents at scale. The position requires North American work authorization and focuses on distributed systems, ML infrastructure, and developer experience. Cohere encourages applicants who do not match every criterion to apply.

Responsibilities

Build secure execution environments for agent-generated code

Design identity and authentication systems for AI agents

Develop model routing and orchestration across different model types and environments

Implement rate limiting, quotas, and resource management for agent workflows

Create state management, memory, and filesystem abstractions for agents

Prototype and evaluate new technologies for production infrastructure

Partner with research teams to shape infrastructure based on future agent system needs

Qualifications

Experience building production ML infrastructure with strong systems fundamentals

Hands-on work with agentic systems, multi-agent workflows, or agent development frameworks

Familiarity with model routing and LLM provider frameworks across different model types and environments

Experience with scalable, fault-tolerant distributed systems and Kubernetes

Proven track record of moving quickly on prototypes and making valuable decisions about productionization

Application conditions

Valid work authorization required for listed locations (Toronto, Canada; United States)

Hybrid attendance may be required at Cohere offices in Toronto, London, New York City, San Francisco, Montreal, Paris, Berlin, or Seoul





Occasional travel may be required for annual company offsite and office visits

Remote eligibility is not worldwide; must comply with regional residency and authorization requirements

Preferred qualifications

Experience across on-prem, private cloud, and public cloud environments

Familiarity with storage systems, embedded databases, or filesystem abstractions

Experience with code execution sandboxes such as gVisor, Firecracker, Kata, or WASM runtimes

Interest in emerging ML infrastructure, edge inference, or browser-native models

Open-source contributions to LLM or agent infrastructure projects

Experience with identity, workload auth, or capability-based security systems

Compensation and benefits

$240K - $360K USD per year salary

Weekly lunch stipend of $75/£75 or equivalent in local currency

Full health and dental benefits with separate budget for mental health

RRSP matching, 401K, Pension Scheme

100% Parental Leave top-up for up to 6 months for either parent

Annual enrichment benefits including Arts & culture, fitness/wellness, quality time, and workspace improvement credit

Education & learning stipend for conferences, courses, and coaching

6 weeks of paid vacation (30 working days)

Budget for traveling to other offices if remote, plus annual company offsite





$500 home office stipend to set up workspace properly

What this role focuses on

Build and maintain the distributed infrastructure platform that enables autonomous AI agents at scale, including secure code execution environments, identity and authentication systems, model routing and orchestration, and resource management for long-running agent workflows. Work at the intersection of ML systems, distributed infrastructure, and developer experience to turn research prototypes into production-ready infrastructure.

Employer-published remote role. Listed location(s): Toronto; Canada; United States. Remote does not imply worldwide eligibility. Check the original listing for residency, work authorization, time zone, language and travel requirements.

Who are we? Cohere is the leading security-first enterprise AI company. We build cutting-edge foundation AI models and end-to-end products that are designed to solve real-world business problems.

We’re training and deploying frontier models for enterprises who are building AI systems. We believe that our work is instrumental to the widespread adoption of AI and we are looking for folks that want to be part of that.

We obsess over what we build. Each one of us is responsible for contributing to increasing the capabilities of our models and the value they drive for our customers. Cohere is a team of researchers, engineers, designers, and more, who are all passionate about their craft.

We are a global technology company headquartered in Toronto with key offices in London, New York City, San Francisco, Montreal, Paris, Berlin and Seoul. Join us

About the role. We’re building the next ge

📌 Staff/Senior Staff Software Engineer, Agent Infrastructure (Toronto)
🏢 PromptCube
📍 Toronto