About

Accelyst is an innovative AI Consultancy delivering secure, integrated, and ROI-focused technology solutions. We combine industry expertise with leading-edge technology to help organizations modernize operations, improve efficiency, and accelerate transformation.

Data Analytical Specialist/Scientist

Onsite 5 days a week in Toronto, Ontario

Working Hours: 7.2hrs

Role Overview

We are seeking a Data Scientist to support enterprise Data & AI initiatives within a modern cloud-based data platform. The role involves supporting machine learning and large language model (LLM) use cases, executing inference workloads, assisting with experimentation and evaluation processes, and contributing to data preparation and performance analysis.

The ideal candidate will have experience with Azure data technologies, Databricks, MLflow, Python, and exposure to LLM frameworks such as DSPy, LangChain, or Hugging Face.

Key Responsibilities

Inference Execution & Monitoring

- Run and monitor batch inference jobs in Databricks and other cloud-based environments.
- Troubleshoot inference issues and document findings.
- Track inference results and escalate model performance anomalies when required.
- Maintain logs and reports for model execution activities.

Pipeline & Model Evaluation Support

- Support the execution of DSPy and other LLM-based pipelines.
- Load and manage custom models, configurations, and evaluation workflows.
- Conduct evaluation runs using established performance metrics.
- Analyze, track, and summarize results from inference and experimentation activities.
- Assist in benchmarking and model comparison exercises.

Data Preparation & Experimentation

- Prepare, clean, and validate datasets for inference and evaluation purposes.




- Annotate model outputs to support error analysis and performance assessment.
- Develop and maintain reproducible Python scripts and notebooks.
- Support experimentation activities and maintain documentation of results and methodologies.

Required Qualifications

Education

- Bachelor's degree (or currently pursuing/completed) in Computer Science, Data Science, Artificial Intelligence, Machine Learning, or a related field.

Technical Skills

- Strong programming skills in Python .
- Experience with Python libraries such as:
- Pandas
- Scikit-learn
- PyTorch
- Experience with Azure data and AI technologies .
- Hands-on experience with:
- Databricks
- MLflow
- Exposure to LLM frameworks such as:
- DSPy
- LangChain
- Hugging Face
- Equivalent AI/LLM frameworks
- Familiarity with machine learning workflows, experimentation, and model evaluation.

Soft Skills

- Solid analytical and problem-solving abilities.
- Detail-oriented with excellent organizational skills.
- Ability to manage and track multiple experiments and deliverables.
- Strong documentation and communication skills.

Must-Have Skills

- Azure Data & AI ecosystem experience.
- Databricks.
- MLflow.
- Python programming.
- Exposure to LLM frameworks (DSPy, LangChain, Hugging Face, or equivalent).
- Experience with Pandas, Scikit-learn, and/or PyTorch.
- Understanding of model inference, evaluation, and experimentation workflows.

Evaluation Criteria

Technical Skills (50%)

- Python programming
- Data science and machine learning fundamentals
- Azure data ecosystem knowledge

LLM Frameworks (25%)

- DSPy
- LangChain
- Hugging Face
- Related LLM technologies

Databricks & MLflow (25%)

- Databricks workflows
- MLflow experiment tracking
- Model deployment and inference operations

📌 Data Analytical Specialist/Scientist (Toronto)
🏢 Accelyst
📍 Toronto