Build AI observability systems at BMO in a hybrid role, using Python, telemetry, and evaluation methods for production models and agents.
As a Senior AI Engineer, you will design systems that trace LLM calls, monitor token usage, detect drift, and evaluate autonomous workflows. The work includes machine learning, deep learning, statistical analysis, and specialized technical support. You will help govern agent behavior and diagnose non-routine production problems.
Key Responsibilities: Design and maintain AI model observability systems
Trace agent trajectories, prompts, tools, and outputs
Build automated evaluations for relevance and hallucinations
Monitor drift, latency, errors, and data quality
Define guardrails and self-remediation boundaries
Requirements: Bachelor's degree in Computer Science or Engineering
Seven to ten years of Site Reliability Engineering experience
Experience in Artificial Intelligence Observability Engineering
Python, SQL, LogQL, or PromQL proficiency
Experience with Prometheus, Grafana, or Datadog
Apply AI engineering, observability, and evaluation expertise to support reliable agent workflows at BMO.

#J-18808-Ljbffr

📌 Senior AI Observability Engineer (Ontario)
🏢 Aimlroles
📍 Ontario