Shape trusted AI evaluation at CrowdStrike using agent frameworks, LLM APIs, and verification-first test automation. Work on cloud-based cybersecurity systems.As a Software Development Engineer in Test, you will help determine how CrowdStrike AI agents are tested, evaluated, and trusted in production. The position combines test automation with eval-driven development across prompts, models, tools, and orchestration.

You will support functional, integration, performance, and end-to-end validation.Key Responsibilities:

Maintain golden datasets and scoring pipelines for AI agentsMeasure correctness, tool use, safety, latency, and costFeed production failures into evaluation setsBuild AI-powered workflows for test data and triageReview agent designs for testability and evaluabilityRequirements:

Computer Science degree or equivalent experienceKnowledge of object-oriented programming and distributed systemsPython preferred for evaluation and agent workExperience testing REST, gRPC, and JSON servicesVerification-first judgment about probabilistic AI outputUse AI evaluation, test automation, Python, and distributed-systems skills to improve trusted production testing at CrowdStrike.

📌 Ai Evaluation Test Automation Engineer (Winnipeg)
🏢 CrowdStrike
📍 Winnipeg