We are seeking an experienced Senior QE Automation Engineer to join our AI Development Lifecycle (AIDLC) team. This role focuses on ensuring quality and reliability of AI/ML systems through comprehensive test automation strategies, specialized AI model validation, and robust quality engineering practices. Key Responsibilities

Test Automation & Strategy Create comprehensive test automation strategies covering unit, integration, system, and end-to-end testing Implement continuous testing practices within CI/CD pipelines for AI model deployment Develop automated tests for model training, inference, and monitoring systems AI/ML Quality Assurance

Validate AI model performance, accuracy, bias, fairness, and robustness Design test cases for model drift detection and data quality validation Implement automated testing for model versioning and A/B testing scenarios Conduct performance and load testing for ML inference endpoints Validate data pipelines, feature engineering, and ETL processes Tools & Infrastructure

Build and maintain test infrastructure for AI workloads Integrate testing tools with MLOps platforms. Implement monitoring and observability for test automation systems Manage test data and synthetic data generation for AI testing Required Qualifications

Technical Skills





5+ years of experience in QA automation engineering 2+ years of hands-on experience testing AI/ML systems or data-intensive applications Solid programming skills in Python (required) and familiarity with other languages (Java, JavaScript, Typescript)

Expertise in test automation frameworks: Pytest, Selenium, Playwright, Cypress, or similar Experience with API testing tools: Postman, REST Assured, or similar Proficiency with CI/CD tools: Jenkins, GitLab CI, GitHub Actions

Strong understanding of ML concepts: model training, evaluation metrics, inference, feature engineering AI/ML Testing Experience

Experience testing machine learning models (classification, regression, NLP) Knowledge of model evaluation metrics (accuracy, precision, recall, etc.) Understanding of data quality testing and validation techniques Familiarity with ML frameworks Experience with model monitoring and observability tools Infrastructure & Cloud

Experience with cloud platforms: AWS (Lambda), Azure ML Knowledge of containerization: Docker, Kubernetes Understanding of distributed systems and microservices architecture Experience with version control systems (Git) and collaborative development workflows API Framework – FastAPI LLM Orchestration – LangGraph Knowledge Graph – Neo4j Vector Store - pgvector

📌 QE Automation Engineer with AI experience (Toronto)
🏢 Socket.dev
📍 Toronto