Cerebras Systems builds the world's largest AI chip, 56 times larger than GPUs. This architecture allows Cerebras to deliver industry-leading training and inference speeds; This order of magnitude increase in speed is transforming the user experience of AI applications, unlocking real-time iteration and increasing intelligence via additional agentic computation. Cerebras works with the leading model labs, global enterprises, and cutting-edge AI-native startups. Cerebras is building a new generation of disaggregated AI inference systems that combine GPU-accelerated prefill with ultra-fast decode on the Cerebras Wafer-Scale Engine.

We are hiring a Software Engineer to productionize and optimize our GPU serving stack, working across our custom inference APIs, the vLLM serving runtime, the AMD ROCm software stack, and rack-scale AMD GPU infrastructure, to make this new serving path reliable, numerically correct, observable, and exceptionally performant.

You will write production code, establish operational practices for a recent accelerator fleet, and drive improvements in time to first token, throughput, tail latency, and capacity efficiency. Design, build, deploy, and maintain the complete GPU prefill path, spanning API services, model-serving workers, vLLM, PyTorch, ROCm, GPU nodes, networking, and rack-scale infrastructure.

Establish deployment, upgrade, rollback, health-checking, capacity-management, and failure-recovery practices for the AMD GPU fleet. Improve fault isolation, graceful degradation, automated recovery, incident response, and post-incident remediation across the serving stack.

Improve inference performance. Optimize model-serving behavior. Diagnose complex failures and performance regressions across application code, vLLM, PyTorch, ROCm/HIP, collective communication libraries, kernels, drivers, firmware,



networking, and hardware.

Build validation and regression infrastructure for model quality, numerical accuracy, precision changes, quantization, determinism, and compatibility across software and hardware releases.

Build performance and correctness infrastructure. 5+ years of software engineering experience, including substantial individual-contributor ownership of complex production systems.

- Experience building, operating, or optimizing production inference systems for large language models, multimodal models, or similarly demanding GPU workloads.
- Strong programming ability in C++ and Python, including experience with multithreading, concurrency, memory management, and performance-sensitive software.
- Hands-on experience with a high-performance model-serving framework such as vLLM, SGLang, TensorRT-LLM, Triton Inference Server, or an equivalent internally developed system.
- Strong understanding of GPU execution and performance, including asynchronous execution, memory movement, synchronization, kernel launches, communication overhead, and profiling methodology.
- Experience debugging distributed systems across multiple layers rather than treating the serving framework or accelerator runtime as a black box.
- Experience with Linux, containers, Kubernetes or comparable orchestration systems, observability, CI/CD, and operating latency-sensitive services in production.
- Ability to design rigorous benchmarks,



interpret noisy performance results, identify bottlenecks, and translate findings into production improvements.
- Strong communication and technical leadership skills, with a demonstrated ability to drive ambiguous cross-functional projects to completion.
- Bachelor’s degree in Computer Science, Computer Engineering, Electrical Engineering, or a related discipline, or equivalent practical experience.

Experience modifying or contributing to vLLM, SGLang, PyTorch, Triton, TensorRT-LLM, or another open-source ML systems project.

Understanding of KV-cache transfer, prefix caching, continuous batching, chunked prefill, request scheduling, and memory-aware admission control.

Experience with multi-GPU and multi-node inference, including tensor parallelism, pipeline parallelism, expert parallelism, RDMA, collective communication, and failure handling.

Experience with reduced-precision inference and quantization formats such as BF16, FP8, FP4, INT8, or INT4, including validation of their numerical and model-quality effects.

Experience building numerical-comparison, determinism, model-validation, or performance-regression test systems.

Experience collaborating directly with accelerator vendors, framework maintainers, or open-source communities.

Build a breakthrough AI platform beyond the constraints of the GPU.

Publish and open source their cutting-edge AI research.

Work on one of the fastest AI supercomputers in the world.

Our simple, non-corporate work culture that respects individual beliefs.

We try every day to build a work environment that empowers people to do their best work through continuous learning, growth and support of those around them.

This website or its third-party tools process personal data.

📌 Software Engineer, Development & Deployment (Toronto)
🏢 Cerebras
📍 Toronto