Cerebras Systems builds the world's largest AI chip, 56 times larger than GPUs. This architecture allows Cerebras to deliver industry-leading training and inference speeds; This order of magnitude increase in speed is transforming the user experience of AI applications, unlocking real-time iteration and increasing intelligence via additional agentic computation. Cerebras works with the leading model labs, global enterprises, and cutting-edge AI-native startups.

Cerebras is building a new generation of disaggregated AI inference systems that combine GPU-accelerated prefill with ultra-fast decode on the Cerebras Wafer-Scale Engine. We are hiring a Senior Software Engineer to build and evolve the ML API layer that makes this heterogeneous serving system accessible, reliable, and easy to use. This role sits at the intersection of machine learning systems, API design, model serving, and distributed systems.

You will translate evolving model capabilities and customer requirements into clear designs, lead implementation across multiple components, and carry features through validation, rollout, and production operation. Your work will help ensure that streaming, sampling, tool use, structured outputs, multimodal inputs, and model configuration behave correctly and consistently. You will work closely with model enablement, compiler, runtime, cloud infrastructure, product, and customer-facing teams, as well as customers directly.

This is a hands-on individual-contributor role with opportunities to lead small project teams, mentor engineers, and improve the quality and reliability of production inference software. Design, implement, test, and operate features for chat completions, text generation, streaming, model configuration, tool calling, structured outputs, multimodal inputs, and emerging inference capabilities. Write clear design documents, evaluate technical tradeoffs, break projects into deliverable milestones, and coordinate implementation with a small group of engineers and partner teams.

Integrate foundation models, tokenizers, chat templates, generation configuration,



sampling methods, and model-specific features into the serving platform. Integrate serving components. Deliver measurable performance improvements.

Own correctness and release quality. Build conformance tests, model-validation suites, numerical comparisons, integration tests, and performance benchmarks that catch regressions in owned features and support safe releases. Improve customer and developer experience.

Improve configuration, documentation, examples, and debugging workflows. Raise the engineering quality bar. 5+ years of software engineering experience, including independent ownership of production services, distributed systems, or substantial software components. ~ Strong programming ability in Python and proficiency in Go, C++, Rust, or another language used to build production services. ~ Experience designing and delivering features that span multiple components, including documenting designs, evaluating tradeoffs, and coordinating implementation with other engineers. ~ Experience building and evolving production APIs, with sound practices for validation, error handling, compatibility, and testing.

Experience integrating software across service, framework, runtime, or infrastructure boundaries. ~ Experience operating services on Linux using containers, Kubernetes or comparable orchestration systems, and CI/CD. ~ Ability to diagnose correctness, reliability, and performance issues across distributed components and deliver measurable improvements. ~ Strong communication and collaboration skills, including experience leading technical delivery for a small project and supporting other engineers through reviews or mentorship.



~ Ability to turn ambiguous requirements into maintainable software while balancing customer needs, delivery timelines, and operational quality. ~ Bachelor’s degree in Computer Science, Computer Engineering, Electrical Engineering, or a related discipline, or equivalent practical experience.

Experience building or operating production ML or LLM inference APIs, including OpenAI-compatible interfaces.

Experience extending or contributing to vLLM, SGLang, PyTorch, Hugging Face Transformers, Triton, TensorRT-LLM, or another ML systems project. Understanding of tokenization, chat templates, generation configuration, logits processing, stopping criteria, tool calling, structured generation, or constrained decoding.

Experience building API conformance, model-quality, numerical-comparison, determinism, or performance-regression test systems.

Experience with multi-model or multi-tenant serving, including routing, admission control, quotas, rate limiting, or capacity-aware scheduling.

Experience developing performance-sensitive or highly concurrent services in Go, C++, Rust, or a similar language. Familiarity with GPU inference, AMD Instinct accelerators, ROCm, CUDA, collective communication, or multi-node model serving.

Experience building SDKs, developer tools, configuration systems, or self-service ML workflows, or collaborating directly with customers and open-source communities. People who are serious about software make their own hardware. At Cerebras, we have built a breakthrough architecture that is unlocking new opportunities for the AI industry.

Build a breakthrough AI platform beyond the constraints of the GPU. Publish and open source their cutting-edge AI research. Work on one of the fastest AI supercomputers in the world. Our easy, non-corporate work culture that respects individual beliefs.

We try every day to build a work environment that empowers people to do their best work through continuous learning, growth and support of those around them. This website or its third-party tools process personal data.

📌 Senior Software Engineer, Ml Api - Ai Inference (Toronto)
🏢 FDE Times
📍 Toronto