Overview The AI Engineer V (Principal) is the most senior individual-contributor technical authority for artificial intelligence and machine learning within AssistRx Engineering. Reporting to the Vice President, Engineering, this role defines and drives the company’s AI technical strategy and leads the design, build, and deployment of production-grade AI solutions across the specialty medication access ecosystem, including large language model applications, predictive models, intelligent document processing, and agentic automation that improve speed, accuracy, and scale in patient onboarding, benefits verification, prior authorization, document intake, and patient and provider engagement.
The AI Engineer V (Principal) shapes AI direction at the enterprise level, solves the company’s most complex and ambiguous technical problems, and sets the architecture, evaluation standards, and responsible AI practices for all AI work at AssistRx. The role serves as a trusted advisor to engineering and executive leadership, represents AssistRx’s AI capabilities with clients, partners, and auditors, and develops the next generation of AI technical leaders. The Principal AI Engineer partners closely with Software Engineering, Product, Architecture, Data, Security, Compliance, and Client Services, and is accountable for the quality, performance, cost efficiency, and safe and compliant use of AI with protected health information across the enterprise.
Responsibilities AI Strategy & Solution Design Define the multi-year AI technical strategy and roadmap in partnership with the Vice President, Engineering, Product, and Architecture, aligning AI investments to business value, feasibility, and risk
Lead the design of the most complex enterprise-scale AI and machine learning solutions, including generative AI, predictive models, and intelligent document processing that span multiple platforms, products, and clients
Architect large language model applications using retrieval augmented generation, prompt engineering, structured outputs, function and tool calling, and agent-based orchestration
Set the patterns for agentic workflows, including task decomposition, tool and function calling, memory and state management, routing, clear boundaries, human review checkpoints, and fallback paths for high-impact decisions such as benefits verification and prior authorization
Identify new AI opportunities and emerging technologies that create competitive advantage, and lead proofs of concept that move proven pilots to scale
Determine the right approach for each problem, including prompting, retrieval augmentation, tool use, task-specific models, and conventional software or deterministic logic when a model is not the right answer
Set the standard for design documentation of AI features, covering intended behavior, context and retrieval design, prompt and agent structure, evaluation approach, and fallback behavior
AI Foundations & Platform Own the enterprise AI reference architecture, platform strategy, and engineering standards, including model access, prompt and version management, and retrieval services, and ensure their adoption across all engineering teams
Lead evaluation of commercial models, AI platforms, and vendor tools, and make build-versus-buy recommendations to executive leadership based on quality, cost, security, strategic fit, and time to value
Guide vendor and cloud provider relationships from a technical perspective, including model selection, capacity, pricing, and risk
Anticipate scaling, reliability, and cost challenges in AI systems, and lead architectural investments that address them before they affect clients
Model Development, Evaluation & Optimization Set organization-wide targets for accuracy, groundedness, hallucination rate, task completion, latency, and cost, and guide model selection, tuning, and evaluation against them
Define the evaluation strategy, benchmark datasets, and automated test frameworks used across the organization to measure output quality and detect regressions before each release
Guide optimization of retrieval pipelines, embeddings, chunking strategies, and inference performance to improve quality and reduce cost per transaction at scale
Direct error analysis and human-in-the-loop review programs with operations and clinical subject matter experts, and ensure findings drive improvement of prompts, models,



and guardrails
Establish the experimentation approach, including A/B testing across prompts, retrieval configurations, model selection, and workflow design
Define and enforce quality gates and regression thresholds so AI features must meet defined accuracy and safety criteria before release
Data Engineering & Integration Set the technical direction for the pipelines that prepare structured and unstructured healthcare data, such as enrollment forms, payer documents, and clinical notes, for training, retrieval, and inference
Partner with Data Engineering and Architecture to define enterprise data contracts, feature sets, vector stores, and lineage for AI workloads
Define enterprise standards for de-identification, masking, and data minimization when working with protected health information
Machine Learning Operations & Production Support Define enterprise standards for CI/CD, source control, containerization, and infrastructure-as-code practices for AI workloads
Establish versioning, promotion, and rollback practices for prompts, agents, and model configurations, treating them as versioned artifacts alongside application code
Set the observability strategy for accuracy drift, groundedness, task completion, token consumption, latency, cost, and reliability, including alerting and incident response practices
Own the strategy for model and inference spend, applying caching, routing, and model right-sizing to keep AI solutions cost-effective at scale
Serve as the final technical escalation point for critical production issues and client-reported defects in AI-enabled features, and drive systemic improvements that prevent recurrence
Responsible AI, Security, Privacy & Compliance Lead the company’s responsible AI approach and ensure solutions satisfy healthcare privacy, data security, and regulatory requirements, including HIPAA and HITRUST
Partner with Security, Compliance, and Legal to address AI-specific risks such as prompt injection, data leakage, and insecure output handling, and to confirm vendor agreements (including Business Associate Agreements) are in place before PHI is processed
Define guardrails, content filtering, human oversight, and audit trails appropriate to the risk profile of each use case
Contribute to internal AI governance, and represent AssistRx’s AI practices in audits, client due diligence, and security reviews
Track emerging AI regulation and industry guidance, and translate it into engineering policy and practice
Technical Leadership, Influence & Enablement Serve as a trusted technical advisor to the Vice President, Engineering and executive leadership on AI strategy, risk, investment, and capability
Provide technical leadership across the engineering organization, influencing architecture and engineering direction at the enterprise level without direct authority
Mentor and develop Staff, Senior, and mid-level engineers, build the AI technical bench, and raise AI fluency across engineering and the broader company
Lead cross-functional technical initiatives and resolve complex technical disagreements across teams
Work with Product Management, Client Services, and business leaders to identify high-value AI use cases and define clear success measures before build begins
Communicate AI capabilities, limitations, risks, and results in plain language to technical and non-technical audiences, including executives, clients, and external partners
Contribute to hiring strategy, interviewing, and role definition for AI and related engineering roles
Advance AI-assisted and agentic development practices across the engineering organization, including specification-driven workflows and agentic coding tools, and share what works with engineering peers
Participate in agile team activities as needed, including sprint reviews and retrospectives, and conduct regular self-guided study to stay at the forefront of a fast-moving AI landscape
Qualifications Bachelor’s degree in Computer Science,



Engineering, Data Science, or related technical field (or equivalent experience); Master’s degree preferred
10+ years of software engineering experience, including 6+ years building and deploying AI or machine learning solutions in production
Track record of leading the strategy and delivery of enterprise-scale AI solutions used by real customers or operations teams, with significant, measurable business outcomes
Expert programming skills in Python, along with experience in .NET/C#/Java or a comparable enterprise language, including building backend services and REST APIs (for example FastAPI, Flask, or ASP.NET)
Deep, hands‑on expertise with large language models and generative AI patterns, including retrieval augmented generation, prompt engineering, embeddings, vector and hybrid search, reranking, structured outputs, function and tool calling, context window management, and model evaluation
Extensive experience architecting and operating production AI workloads on cloud-based AI and machine learning platforms (AWS preferred, including services such as Amazon Bedrock and Amazon SageMaker; equivalent Azure experience with Azure AI Foundry, Azure OpenAI Service, or Azure Machine Learning also acceptable)
Expert knowledge of APIs, microservices, distributed systems, SQL, and modern development practices including Git, CI/CD, containerization, and automated testing
Demonstrated ability to deliver in an workplace where data privacy, security, and regulatory requirements shape solution design
Demonstrated enterprise-level technical leadership, including setting technical strategy, driving architectural decisions across teams, and developing other technical leaders
Ability to work independently on highly ambiguous, high-impact problems, set direction, and make sound tradeoffs with limited precedent
Hands‑on experience building software with agentic coding tools such as Claude Code, GitHub Copilot agents, Codex, or Cursor, including specification-driven workflows and multi‑step agent execution
Deep practical understanding of how large language models behave in production, including hallucination, prompt sensitivity, non‑determinism, and latency and cost tradeoffs
Expert knowledge of evaluation methods for LLM systems, including automated scoring, LLM‑as‑judge techniques, and human review workflows
Exceptional written and verbal communication skills, with the ability to influence executives, clients, and engineers and to represent the company’s AI approach externally
Preferred Qualifications Master’s or doctoral degree in Computer Science, Machine Learning, Data Science, or related field
Experience in healthcare technology, specialty pharmacy, HUB services, ePA, or EHR/EMR integrations
Experience with intelligent document processing, OCR, and data extraction from unstructured enrollment or clinical documents
Applied machine learning or data science background, including experiment design, model evaluation, and error analysis
Experience with Docker, Kubernetes, or infrastructure-as-code at scale
Experience with agent and orchestration frameworks such as LangChain, LangGraph, LlamaIndex, Semantic Kernel, Strands, or Model Context Protocol
Experience with machine learning operations tooling, model registries, feature stores, and LLM observability or evaluation platforms
Experience with model fine‑tuning or smaller task‑specific models for classification and extraction
Familiarity with AI security guidance such as the OWASP Top 10 for LLM Applications
Familiarity with HIPAA, HITRUST, or responsible AI frameworks such as the NIST AI Risk Management Framework
Prior experience establishing an AI function or serving as the senior AI technical leader for an organization
Experience representing technical capabilities in client due diligence, security reviews, or external audits
COMPETENCIES Technical Depth
Strategic Thinking
Analytical Rigor & Problem Solving
Experimentation & Measurable Outcomes
Accuracy & Quality
Sound Judgment & Risk Awareness
Ownership & Accountability
Pragmatism & Bias for Delivery
Communication
Collaboration & Influence
Technical Leadership & Talent Development
Curiosity & Learning Agility
Adaptability
Initiative & Innovation
Ethics & Integrity
Pay Range USD $162,239.00 - USD $204,048.00 /Yr.

#J-18808-Ljbffr

📌 AI Engineer V (Principal) (Manitoba)
🏢 AssistRx
📍 Manitoba