We are seeking an experienced Senior QE Automation Engineer to join our AI Development Lifecycle (AIDLC) team. This role focuses on ensuring quality and reliability of AI/ML systems through comprehensive test automation strategies, specialized AI model validation, and robust quality engineering practices.

Key Responsibilities

- Test Automation & Strategy
- Create comprehensive test automation strategies covering unit, integration, system, and end-to-end testing
- Implement continuous testing practices within CI/CD pipelines for AI model deployment
- Develop automated tests for model training, inference, and monitoring systems

AI/ML Quality Assurance

- Validate AI model performance, accuracy, bias, fairness, and robustness
- Design test cases for model drift detection and data quality validation
- Implement automated testing for model versioning and A/B testing scenarios
- Conduct performance and load testing for ML inference endpoints
- Validate data pipelines, feature engineering, and ETL processes

Tools & Infrastructure

- Build and maintain test infrastructure for AI workloads
- Integrate testing tools with MLOps platforms.
- Implement monitoring and observability for test automation systems
- Manage test data and synthetic data generation for AI testing

Required Qualifications

Technical Skills

- 5+ years of experience in QA automation engineering




- 2+ years of hands-on experience testing AI/ML systems or data-intensive applications
- Strong programming skills in Python (required) and familiarity with other languages (Java, JavaScript, Typescript)
- Expertise in test automation frameworks: Pytest, Selenium, Playwright, Cypress, or similar
- Experience with API testing tools: Postman, REST Assured, or similar
- Proficiency with CI/CD tools: Jenkins, GitLab CI, GitHub Actions
- Solid understanding of ML concepts: model training, evaluation metrics, inference, feature engineering

AI/ML Testing Experience

- Experience testing machine learning models (classification, regression, NLP)
- Knowledge of model evaluation metrics (accuracy, precision, recall, etc.)
- Understanding of data quality testing and validation techniques
- Familiarity with ML frameworks
- Experience with model monitoring and observability tools

Infrastructure & Cloud

- Experience with cloud platforms: AWS (Lambda), Azure ML
- Knowledge of containerization: Docker, Kubernetes
- Understanding of distributed systems and microservices architecture
- Experience with version control systems (Git) and collaborative development workflows
- API Framework – FastAPI
- LLM Orchestration – LangGraph
- Knowledge Graph – Neo4j
- Vector Store - pgvector

📌 QE Automation Engineer with AI experience (Toronto)
🏢 Socket.dev
📍 Toronto