Senior Artificial Intelligence Evaluation Engineer
QA Go

Important: if an employer asks you to log into their system via iCloud or Google, send a code, an SMS or Telegram password, run some code, or install software - refuse. These are signs of fraud.

is looking for a Senior AI Evaluation Engineer to join our team full-time in the US.

We are a custom product engineering company that supports both multinational organizations and scaling startups to solve their most complex business challenges.

About the role:

As a Senior AI Evaluation Engineer, become a part of a cross-functional development team engineering experiences of tomorrow.

Responsibilities:

Build the central eval harness and templates every pod uses; hub-and-spoke: central standards, pod-written tests

Create golden datasets with business and clinical reviewers, including safety and medical-accuracy suites

Set and defend pass thresholds; audit pod evals; report eval pass rates and incidents monthly

Stand up production monitoring for drift and regression with the pipeline engineer

Requirements:

4+ years in ML/LLM evaluation, QA engineering for AI systems, or applied research engineering

Hands-on with eval frameworks, LLM-as-judge patterns and their failure modes, statistical rigor on small samples

Independent spine: the team shipping a thing does not set its own pass bar





Healthcare or safety-critical evaluation experience; red-teaming background

What's in it for you?

Strong community: Work alongside top professionals in a friendly, open-door environment

Growth focus: Take on large-scale projects with a global impact and expand your expertise

Tailored learning: Boost your skills with internal events (meetups, conferences, workshops), Udemy access, language courses, and company-paid certifications

Endless opportunities: Explore diverse domains through internal mobility, finding the best fit to gain hands-on experience with cutting-edge technologies

Care: Healthcare, Basic Life Insurance, Short and Long-term disability insurance according to the Company's Benefit Plans

About us:

At , we are always exploring innovations, empowering each other to achieve more, and engineering solutions that matter. With us, you'll work with cutting-edge technologies, contribute to impactful projects, and be part of a One Team culture that values collaboration and progress. In the US, is growing quick - inviting experienced professionals to lead digital transformation alongside Fortune 500 clients. Be part of a company where innovation and impact go hand in hand.

The range shows the lower and upper market bands: 25% of vacancies pay below the first value, the median is the typical salary, and 75% marks the upper band.

#J-18808-Ljbffr

📌 Senior Artificial Intelligence Evaluation Engineer (Manitoba)
🏢 Talanto
📍 Manitoba