Test Engineer to develop automated testing frameworks using Databricks and Spark Our financial services client is seeking a Test Engineer - Data (10+ years) to develop automated testing frameworks using Databricks and Spark

Join a Quality Assurance team focused on validating Databricks-based solutions and large-scale data pipelines within a Lakehouse architecture. This role supports automated testing, data quality validation, Spark job testing, and monitoring initiatives across ETL/ELT workflows and Delta Lake environments. The position works closely with data engineering teams to ensure data accuracy, reliability, and performance across data processing pipelines.

Permanent, 3 days a week onsite downtown Toronto, Working Hours: EST

Must Haves

- 10+ years as a Data Test Engineer
- 2+ years hands-on experience with Databricks and Spark
- Robust programming skills in Python (PySpark) and SQL
- Experience with AWS services including S3, Glue, and Lambda, plus Delta Lake and Lakehouse architecture




- Proficiency with AI coding agents such as Claude Code , Cursor, Kiro, and Git
- Bachelor’s degree in Computer Science, Data Science, or related field

Nice to Have

- Relevant certifications in Databricks or data testing
- Experience with Agile methodologies
- Knowledge of data governance and compliance requirementsExperience with large-scale data processing

Responsibilities

- Automate testing for ETL/ELT processes in Databricks
- Design and implement automated testing for Databricks notebooks and workflows
- Create test frameworks for Delta Lake tables and ACID transactions
- Develop automated validation for structured streaming pipelines
- Implement automated data reconciliation processes
- Create automated test dashboards
- Generate quality metrics and testing reports

CAD $105,000 to CAD $135,000

Disclaimer: AI may be used in evaluating candidates. This posting is for an existing vacancy.

📌 Test Engineer to develop automated testing frameworks using Databricks and Spark (Toronto)
🏢 S.I. Systems
📍 Toronto