Elevate data quality as an ETL QA Engineer specializing in Databricks and PySpark. This position emphasizes AI-assisted development and efficient data pipeline validation. We are seeking a highly skilled ETL QA Engineer with extensive experience in Databricks and expert-level PySpark capabilities. You'll focus on ensuring the integrity of large-scale data pipelines and leverage tools such as GitHub Copilot and AI agents to increase testing efficiency.

Your role will also include contributing to AI-driven knowledge bases and fostering a robust testing culture. Key Responsibilities:

- Validate large-scale data pipelines for quality assurance
- Utilize PySpark for transformations and optimization techniques
- Implement CI/CD using GitHub Copilot and actions




- Construct effective prompts for testing with AI tools
- Document complex data validation processes clearly Requirements:
- Expertise in Databricks notebooks, jobs, and Delta Lake
- Proven proficiency in automated testing frameworks
- Solid understanding of ETL/ELT concepts and data warehousing
- Experience with agent-based architectures in QA
- Detail-oriented approach to problem-solving Drive data excellence through innovative QA practices and leverage AI tools to enhance efficiency.

📌 ETL QA Engineer with Databricks Expertise (Toronto)
🏢 Kumaran Systems
📍 Toronto