Key Responsibilities

Run and Monitor Inferences

- Execute batch inference jobs on Databricks and other environments.

- Troubleshoot inference issues, log results, and escalate model anomalies.

Pipeline & Evaluation Support

- Assist with running DSPy pipelines and loading custom models/configs.

- Support evaluation runs using internal metrics

- Track and summarize performance metrics for inference experiments.

Data Handling & Experimentation

- Help prepare and clean legal text datasets for inference and evaluation.

- Annotate outputs to support error analysis and benchmarking.

- Maintain reproducible scripts/notebooks for inference runs.

Qualifications

- Background in Computer Science, Data Science, or AI/ML (student or recent grad is good).

- Familiarity with Python and libraries like pandas, scikit-learn,



or PyTorch.

- Exposure to LLM frameworks (DSPy, LangChain, Hugging Face, or equivalent).

- Comfortable working with Databricks / MLflow or other cloud ML environments.

- Detail-oriented with solid organizational skills for tracking experiments.

Evaluation:

- Technical Skills - 50%

- LLM frameworks - 25%

- Databricks/MLflow - 25%

Must have:

- must have experience Azure data tools - azure, databricks, MLflow, python

- Exposure to LLM frameworks (DSPy, LangChain, Hugging Face, or equivalent).

- Familiarity with Python and libraries like pandas, scikit-learn, or PyTorch.,

📌 AI/ML Engineer (Toronto)
🏢 BuzzClan
📍 Toronto