We are looking for an ETL QA Engineer with strong experience in Databricks and Py Spark, combined with hands-on exposure to AI-assisted development and prompt engineering. The ideal candidate will validate large-scale data pipelines, ensure data quality, and leverage tools like Git Hub Copilot, Git Hub Actions/Runners, and AI agents to improve testing efficiency and coverage. You will also help build and validate AI-powered knowledge bases. Required Skills & Experience Core Technical Skills Strong hands-on experience in Databricks (notebooks, jobs, clusters, Delta Lake, SQL). Expert-level Py Spark skills, including transformations, actions, window functions, and optimization techniques. Solid understanding of ETL/ELT concepts, data warehousing, and data modeling (star/snowflake schemas, dimension/fact tables). Proven experience in data validation, data quality checks, and reconciliation techniques. Practical experience with Git Hub, Git Hub Copilot, and Git Hub Actions/Runners for CI/CD pipelines. AI & Prompt Engineering Experience crafting effective prompts for large language models (LLMs) for coding, testing, and documentation. Exposure to agent-based architectures or tools (e.G., workflow/agent frameworks that orchestrate multi-step AI tasks).



Familiarity with AI-based tools for code analysis, test generation, or knowledge management. Knowledge Base & Documentation Experience building AI-driven knowledge bases or documentation systems (e.G., using vector search, embeddings, or LLM-based retrieval). Strong documentation skills, with the ability to translate complex data/QA concepts into clear knowledge articles. Testing & QA Practices Strong understanding of QA methodologies: test planning, test design, defect management, and regression testing. Experience with automated testing frameworks (Python-based testing libraries such as pytest, unittest, or similar). Familiarity with data quality tools/practices (e.G., validation rules, thresholds, anomaly detection). Key Attributes Strong analytical and problem-solving skills with a detail-oriented mindset. Passion for data quality, automation, and continuous improvement. Ability to work in an agile, fast-paced environment and collaborate across multiple teams like Business and operations team and AD team. Curiosity and openness to adopting recent AI tools and practices for QA. #J-18808-Ljbffr

📌 Etl Testing (Toronto)
🏢 Kumaran Systems
📍 Toronto