*hiring on behalf of our client Title: Senior ML Inference / Systems Engineer Time: Full time

Location:
Remote (Canada) Compensation: $250,000
- $300,000 USD / year + competitive equity The Company Our client is a high-growth research lab and inference provider focused on real-time AI for consumer-facing applications. They build first-party speech models, serve LLMs, and operate the infrastructure behind modular APIs designed for high-volume, real-time workloads. Their platform serves over 10 trillion LLM tokens monthly to hundreds of millions of users across companions, healthcare, education, and media. Backed by top-tier venture capital firms, they collaborate with leading technology partners to power sub-second multimodal experiences at global scale. Role Overview Our client is seeking an experienced Senior ML Inference / Systems Engineer to optimize, scale, and maintain high-throughput, low-latency machine learning serving systems. In this role, you will take ownership of the full model deployment lifecycle, from containerizing and profiling models to implementing cutting-edge acceleration techniques across distributed GPU clusters. You will work in a fast-paced, high-impact environment focused on solving latency, throughput, and distributed systems challenges in production.

Key Responsibilities

• Design, build, and optimize high-performance inference pipelines for large-scale production workloads
• Implement advanced model acceleration techniques including quantization, distillation, continuous batching, paged attention, and speculative decoding




• Profile and optimize low-level code to maximize performance and efficiency across NVIDIA GPUs
• Scale distributed inference infrastructure using modern serving frameworks, Kubernetes, Ray, and custom load-balancing solutions
• Containerize models and collaborate closely with research teams to deploy stable, high-throughput systems into production
• Troubleshoot and resolve complex system bottlenecks to ensure high availability and sub-second latency across multi-GPU and multi-node setups
• Write clean, high-performance systems code and contribute to core engineering architecture decisions Qualifications
• 5+ years of experience as an ML inference engineer, systems engineer, or senior backend engineer working with high-performance production systems
• Deep technical expertise in modern inference serving frameworks such as vLLM or TRT-LLM
• Proficiency in systems programming and GPU acceleration using C++, CUDA, Rust, or highly optimized Python
• Hands-on experience deploying and managing distributed systems at scale using Kubernetes, Ray, and Docker
• Demonstrated background implementing model acceleration strategies such as caching, quantization, distillation, or speculative decoding
• Strong foundation in systems engineering, low-latency architectures, and multi-GPU/multi-node scaling
• Bachelor’s, Master’s, or PhD in Computer Science, Mathematics, Physics, or equivalent practical systems engineering experience
• Excellent problem-solving skills with a strong bias for shipping stable, high-impact solutions in ambiguous environments

📌 Senior ML Inference (Canada)
🏢 Talently
📍 Canada