We are seeking a detail-oriented ETL Automation Quality Engineer to ensure the reliability| accuracy| and efficiency of our data pipelines.
This role combines deep technical expertise in ETL processes| automation| and quality engineering to validate data integrity and system performance in a cloud-based setting.
Key Responsibilities
Design| develop| and maintain automated test frameworks for ETL workflows| ensuring data accuracy| consistency| and compliance with business requirements.
Write complex SQL queries to validate data transformations| integrity| and quality across databases and data lakes (e.g.| Hadoop| Parquet).Implement Python scripts to automate testing| monitor pipeline performance| and generate reports on data quality metrics.
Collaborate with engineering teams to integrate automated tests into CI/CD pipelines using GitHub and GitHub Actions.
Manage data storage and retrieval processes in AWS S3| ensuring scalability and security.Apply QE methodologies (e.g.| test planning| risk analysis| defect tracking) to identify and resolve issues in ETL pipelines.
Develop and execute test cases for API integrations (nice to have) and validate end-to-end data workflows.
Document test strategies| results| and recommendations for process improvements.Work with cross-functional teams to troubleshoot data discrepancies and optimize ETL processes.
Must-Have Qualifications
Technical Skills:
oExpertise in SQL for data validation and complex querying.
oProficiency in Python for scripting and automation.
oHands-on experience with Unix/Linux| cloud platforms (AWS)| Hadoop| Parquet| and AWS S3.
oFamiliarity with GitHub for version control and collaboration.
oProven experience in automated ETL testing frameworks.
Soft Skills:
oStrong communication skills to articulate technical issues and solutions to diverse stakeholders.
oAnalytical mindset with a focus on quality and attention to detail.
Methodologies:
oUnderstanding of QE principles| including test design| execution| and reporting.
Nice-to-Have
Qualifications Experience with GitHub Actions for CI/CD pipeline automation.
Knowledge of automated API testing tools (e.g.| Postman| REST-assured).
Familiarity with AI/ML tools (e.g.| Copilot) for code optimization or data quality enhancements.
Basic understanding of machine learning concepts as they relate to data pipelines.
Skills
Category
Name
Required
Importance
Experience
SkillCategoryTest1_MN
Digital : Python
Yes
1
> 7 years
📌 CAN_Developer (Toronto)
🏢 VARITE
📍 Toronto
Reply to this offer
Impress this employer describing Your skills and abilities, fill out the form below and leave Your personal touch in the presentation letter.