28 Oct
|
Kumaran Systems
|
Toronto
28 Oct
Kumaran Systems
Toronto
Role Overview We are looking for an ETL QA Engineer with strong experience in
Databricks and PySpark, combined with hands-on exposure to AI-assisted development and prompt engineering.
The ideal candidate will be responsible for validating large-scale data pipelines, ensuring data quality, and leveraging tools like Git
Hub Copilot, Git
Hub Actions/Runners, and AI agents to improve testing efficiency and coverage.
You will also contribute to building and validating AI-powered knowledge bases.
Required Skills & Experience Core Technical Skills Strong hands-on experience in Databricks (notebooks, jobs, clusters, Delta Lake, SQL).
Expert-level PySpark skills, including transformations, actions, window functions, and optimization techniques.
Solid understanding of ETL/ELT concepts , data warehousing, and data modeling (star/snowflake schemas, dimension/fact tables).
Proven experience in data validation , data quality checks, and reconciliation techniques.
Practical experience with Git
Hub , Git
Hub Copilot , and Git
Hub Actions/Runners for CI/CD pipelines. AI & Prompt Engineering Experience crafting effective prompts for large language models (LLMs) for coding, testing, and documentation.
Exposure to agent-based architectures or tools (,
workflow/agent frameworks that orchestrate multi-step AI tasks).
Familiarity with AI-based tools for code analysis, test generation, or knowledge management.
Knowledge Base & Documentation Experience building AI-driven knowledge bases or documentation systems (, using vector search, embeddings, or LLM-based retrieval).
Strong documentation skills, with the ability to translate complex data/QA concepts into clear knowledge articles.
Testing & QA Practices Strong understanding of QA methodologies: test planning, test design, defect management, and regression testing.
Experience with automated testing frameworks (Python-based testing libraries such as pytest, unittest, or similar).
Familiarity with data quality tools/practices (, validation rules, thresholds, anomaly detection).
Key Attributes Strong analytical and problem-solving skills with a detail-oriented mindset.
Passion for data quality, automation, and continuous improvement.
Ability to work in an agile, quick-paced environment and collaborate across multiple teams like Business and operations team and AD team.
Curiosity and openness to adopting new AI tools and practices for QA.
📌 ETL QA (Toronto)
🏢 Kumaran Systems
📍 Toronto