Key Responsibilities

- Build and operate observability for

LLM gateways, model routing, MCP servers, agent runtimes, and AI platform services

.

- Implement end-to-end telemetry using

OpenTelemetry, Langfuse, metrics, logs, and distributed tracing

.

- Monitor and analyze

LLM/agent latency, errors, token usage, model performance, quality, evaluation results, and production behavior

.

- Develop dashboards, alerts, and reporting for

AI reliability, performance, quality, and operational health

.

- Build

AI cost and usage observability

, including attribution by application, team, user, model, workflow, and workplace across providers such as OpenAI, Anthropic, and Gemini.

- Establish observability and evaluation standards for

LLM and agentic applications

, including traces, prompts, responses, tool calls, evaluations, and regression signals.

- Support

AI evaluation and monitoring through golden datasets, scoring, prompt/model comparisons,



and production quality checks.

- Operate AI platform services on

Kubernetes using Helm, Kustomize, GitOps, autoscaling, and progressive/zero-downtime deployments.

- Implement platform security, access control, secrets management, PII protection, guardrails, and audit logging

.

Key Skills

- AI/LLM Observability:

Langfuse, OpenTelemetry, LLM tracing, AI evaluation

- AI Platforms:

LLM gateways, model routing, MCP, agentic AI

- Cloud/Platform:

Kubernetes, Helm, Kustomize, GitOps, APIs

- Monitoring:

Prometheus, Grafana, Datadog or similar platforms

- AI/ML:

OpenAI, Anthropic, Gemini, LangChain, LangGraph, CrewAI, Google ADK

- Engineering:

Python/Java/Go, REST APIs, CI/CD

- Security:

OAuth2/OIDC, JWT, RBAC, secrets management, PII/AI guardrails

📌 Role: AI Platform DevOps Engineer (Canada)
🏢 Triunity Software
📍 Canada