About the company Our client is a software company. The role Raydar is recruiting for this role on behalf of our client. Own the path that takes trained machine learning models into reliable production serving, with a focus on speed, efficiency and dependability at high request volume. The work combines performance tuning of models and GPU-based code with the design of distributed serving systems that run across many machines. What you'll do - Optimize serving latency and throughput for machine learning models. - Take models from research through containerization and into reliable production deployment. - Profile and tune code to make efficient use of GPU hardware. - Design and run distributed serving systems across multiple GPUs and nodes. - Apply model acceleration methods such as quantization, batching and caching. Requirements What we're looking for - Senior-level experience, comparable to a senior software or inference engineer at a large technology company. - Track record of developing server-side or learning-based software and getting it running live for real users. - Hands-on experience with machine learning inference optimization frameworks.
- Proficiency in systems programming and GPU optimization in languages such as C++, CUDA, Rust or optimized Python. - Familiarity with methods that speed up and shrink models, such as weight compression, knowledge transfer between models or token prediction. - Experience with distributed systems, container orchestration and handling high concurrency. - Based in an eligible country for this remote role. Bonus points - Public technical work, such as open-source contributions to inference engines or detailed technical write-ups. - A history of internal progression and promotions within one employer. - A doctorate in a quantitative field, or equivalent hands-on systems experience, with study at a leading university as a further advantage. Benefits Compensation and benefits - Base salary: $250K to $300K - Equity - Annual performance bonus - Health, dental and vision coverage - Flexible time off - 401(k) retirement plan - Health savings account Location and work model - Remote (Canada or UK) - Full-time