Build AI evaluation systems and cost controls for Velixo Intelligence from anywhere in Canada, with priority for Quebec candidates. Python or Type Script skills matter.
As Velixo’s Applied AI Engineer, you will replace anecdotal quality judgments with evidence for MCP-based ERP querying and governed writebacks. You will build regression suites, instrument Gateway actions, and guide model, prompt, and pricing decisions. The work spans LLM APIs, observability, data analysis, and financial systems.
Key Responsibilities: Build regression suites for GIQL, writebacks, and tool selection
Run evaluations before model upgrades and prompt changes
Instrument tokens, caching, models, latency, and tenants
Establish cost-per-action baselines and reduction roadmaps
Supply unit cost data for pricing and allowances
Requirements: Production experience building on LLM APIs
Previous experience creating an evaluation harness
Hands-on familiarity with Langfuse and promptfoo
Comfort with token accounting and caching strategies
Python or Type Script proficiency
Shape Velixo Intelligence through rigorous evaluation, cost engineering, model strategy, and explicit analysis for finance decisions.

📌 Ai Evaluation And Cost Engineer (Montreal)
🏢 Velixo
📍 Montreal