We are seeking an experienced Senior QE Automation Engineer to join our AI Development Lifecycle (AIDLC) team. This role focuses on ensuring quality and reliability of AI/ML systems through comprehensive test automation strategies, specialized AI model validation, and robust quality engineering practices.
Key Responsibilities Test Automation & Strategy
Create comprehensive test automation strategies covering unit, integration, system, and end-to-end testing
Implement continuous testing practices within CI/CD pipelines for AI model deployment
Develop automated tests for model training, inference, and monitoring systems
AI/ML Quality Assurance Validate AI model performance, accuracy, bias, fairness, and robustness
Design test cases for model drift detection and data quality validation
Implement automated testing for model versioning and A/B testing scenarios
Conduct performance and load testing for ML inference endpoints
Validate data pipelines, feature engineering, and ETL processes
Tools & Infrastructure Build and maintain test infrastructure for AI workloads
Integrate testing tools with MLOps platforms.
Implement monitoring and observability for test automation systems
Manage test data and synthetic data generation for AI testing
Required Qualifications Technical Skills **5+ years** of experience in QA automation engineering




**2+ years** of hands-on experience testing AI/ML systems or data-intensive applications
Solid programming skills in **Python** (required) and familiarity with other languages (Java, JavaScript, Typescript)
Expertise in test automation frameworks: Pytest, Selenium, Playwright, Cypress, or similar
Experience with API testing tools: Postman, REST Assured, or similar
Proficiency with CI/CD tools: Jenkins, GitLab CI, GitHub Actions
Strong understanding of ML concepts: model training, evaluation metrics, inference, feature engineering
AI/ML Testing Experience Experience testing machine learning models (classification, regression, NLP)
Knowledge of model evaluation metrics (accuracy, precision, recall, etc.)
Understanding of data quality testing and validation techniques
Familiarity with ML frameworks
Experience with model monitoring and observability tools
Infrastructure & Cloud Experience with cloud platforms: AWS (Lambda), Azure ML
Knowledge of containerization: Docker, Kubernetes
Understanding of distributed systems and microservices architecture
Experience with version control systems (Git) and collaborative development workflows
API Framework – FastAPI
LLM Orchestration – LangGraph
Knowledge Graph – Neo4j
Vector Store - pgvector

#J-18808-Ljbffr

📌 QE Automation Engineer with AI experience (Ontario)
🏢 Socket.dev
📍 Ontario