About the Company An AI and data engineering consultancy that builds data platforms and AI products for enterprise clients. Its delivery team is spread across Europe, and this engagement sits with a major financial services client in the US.

The Role

This is a long-term contract for a senior data scientist to own the training side of a real-time conversational AI system. The system supports front-office staff during live client calls. You will take messy call transcripts and turn them into trustworthy training data, post-train open-weight LLMs, and prove your models beat the current baseline. It suits someone who has trained and evaluated models hands-on, not someone who has only built on top of APIs.

What You Will Do

- Clean and structure large volumes of noisy speech-to-text call transcripts: repair conversation turns, handle redacted text, remove duplicates, and build time-based splits with no leakage
- Build training targets from outcome signals and weak labels, using LLM-assisted annotation checked by domain experts, and balance coverage across roughly 50 objection categories
- Run supervised fine-tuning (LoRA, QLoRA and full-parameter) and preference optimization (DPO, KTO and related methods) on an 8-GPU H100 node, scaling from small prototypes to a large mixture-of-experts model
- Design offline evaluation without gold answers,



including LLM-as-judge rubrics calibrated against expert reviewers and statistically sound agreement checks
- Check whether the original dataset assumptions actually hold against real conversion outcomes, document every experiment, and present findings to technical and business audiences

What You Bring

- 5+ years in machine learning or NLP, including at least 2 years working directly with LLMs
- Hands-on fine-tuning of open-weight models of 7B+ parameters, with practical SFT experience and at least one preference method such as DPO, KTO or ORPO
- Multi-GPU training with DeepSpeed or FSDP on dedicated hardware, plus fluency with Hugging Face Transformers, TRL and PEFT
- Experience processing 1M+ document text datasets and building supervision from incomplete or weak labels
- Strong statistics (confidence intervals, significance testing, confounding), disciplined error analysis, and clean, tested, reproducible Python

Why This Role You will work on real conversation data at enterprise scale, with dedicated H100 compute and a transparent goal: beat the existing fine-tuned model and its best-of-N baseline. The first phase delivers the model itself, and the contract is expected to run long-term as the system takes on more use cases.

📌 Senior Data Scientist – LLM Fine-Tuning (NLP, Contract) (Canada)
🏢 TekRek
📍 Canada