Elevate data quality as an ETL QA Engineer specializing in Databricks and Py Spark. This position emphasizes AI-assisted development and effective data pipeline validation.
We are seeking a highly skilled ETL QA Engineer with extensive experience in Databricks and expert-level Py Spark capabilities. You'll focus on ensuring the integrity of large-scale data pipelines and leverage tools such as Git Hub Copilot and AI agents to increase testing efficiency. Your role will also include contributing to AI-driven knowledge bases and fostering a robust testing culture.

Key Responsibilities
• Validate large-scale data pipelines for quality assurance
• Utilize Py Spark for transformations and optimization techniques




• Implement CI/CD using Git Hub Copilot and actions
• Construct effective prompts for testing with AI tools
• Document complex data validation processes clearlyRequirements:
• Expertise in Databricks notebooks, jobs, and Delta Lake
• Proven proficiency in automated testing frameworks
• Robust understanding of ETL/ELT concepts and data warehousing
• Experience with agent-based architectures in QA
• Detail-oriented approach to problem-solvingDrive data excellence through innovative QA practices and leverage AI tools to enhance efficiency.

📌 Etl qa engineer with databricks expertise (Toronto)
🏢 Kumaran Systems
📍 Toronto