Build AI evaluation systems and cost controls for Velixo Intelligence from anywhere in Canada, with priority for Quebec candidates. Python or TypeScript skills matter.

As Velixo’s Applied AI Engineer, you will replace anecdotal quality judgments with evidence for MCP-based ERP querying and governed writebacks. You will build regression suites, instrument Gateway actions, and guide model, prompt, and pricing decisions. The work spans LLM APIs, observability, data analysis, and financial systems.

Key Responsibilities:

- Build regression suites for GIQL, writebacks, and tool selection
- Run evaluations before model upgrades and prompt changes
- Instrument tokens, caching, models, latency, and tenants
- Establish cost-per-action baselines and reduction roadmaps
- Supply unit cost data for pricing and allowances

Requirements:

- Production experience building on LLM APIs
- Previous experience creating an evaluation harness
- Hands-on familiarity with Langfuse and promptfoo
- Comfort with token accounting and caching strategies
- Python or TypeScript proficiency

Shape Velixo Intelligence through rigorous evaluation, cost engineering, model strategy, and transparent analysis for finance decisions.

📌 AI Evaluation and Cost Engineer (Montreal)
🏢 Velixo
📍 Montreal