About the Role

Join an AI team building systems for physical engineering. You will own the evaluation framework for an engineering AI system, measuring its performance against expert human judgment and guiding improvements in its engineering reasoning.

What You'll Do

- Build an evaluation framework for open-ended engineering design tasks where there may be no single correct answer.
- Work with engineering subject matter experts, customers, partners, and internal teams to understand expert practices and identify system strengths and gaps.
- Recreate real-world engineering tasks and failure modes in evaluation environments that development teams can run repeatedly.
- Design automated judges and human evaluation processes, then assess how well automated results align with expert judgment.
- Translate evaluation findings into priorities for improving engineering reasoning.

What We're Looking For

- Five to fifteen years of experience in AI engineering, AI research, or a related technical role, with experience in physical engineering and/or AI.




- Hands-on experience evaluating agentic AI systems, including creating evaluation tasks, designing automated judges, and comparing automated results with human judgment.
- A background in a physical engineering field such as mechanical, electrical, aerospace, robotics, or power systems, with the ability to assess engineering design quality.
- Experience collaborating directly with customers, subject matter experts, partners, and cross-functional technical teams.
- Strong ownership and communication skills, plus experience building new capabilities in early-stage environments; physical AI or robotics experience is a plus.
- Authorized to work in the United States without visa sponsorship, and able to travel to the San Mateo office for one week every six weeks.

Compensation & Perks

Base salary: $200,000 to $300,000 USD annually. Visa sponsorship is not available.

Location

Remote in Canada, with periodic travel to San Mateo, California.

📌 Member of Technical Staff, AI Evaluation (Canada)
🏢 Wintermeyer Ventures
📍 Canada