Optimize large language model inference with Python and PyTorch across AWS Trainium platforms, including Trainium2 and Trainium3. Improve production AI performance through hands-on accelerator optimization.
As a Senior Machine Learning Systems Engineer, you will improve inference latency, throughput, scalability, and serving costs. The work includes LLM inference frameworks, AWS Neuron SDK, vLLM on Neuron, and production workload optimization. You will evaluate advanced techniques for Qwen-family models.

Key Responsibilities

Optimize decode latency, throughput, and LLM serving costsWork with Trainium2, Trainium3, Neuron SDK,



and vLLMEvaluate speculative decoding and Multi-Token Prediction for Qwen modelsProfile and benchmark GPU and Trainium production workloadsOptimize prefill, decode, batching, KV-cache, and parallelismRequirements:
Five-plus years in Machine Learning or Systems EngineeringTwo-plus years with production LLM inference or servingStrong Python and PyTorch programming experienceExperience with vLLM, TensorRT-LLM, SGLang, or TritonUnderstanding of prefill, decode, batching, and KV-cache managementApply inference performance engineering, Python, PyTorch, and accelerator optimization across demanding production LLM workloads.

📌 Senior Llm Inference Performance Engineer (Canada)
🏢 Aptino
📍 Canada