Apply on Kit Job: kitjob.ca/job/2ghksh

About the Role

We are looking for a hands‑on Data Engineer with deep expertise in Apache Spark and robust programming skills in Python, Scala, and Java. This role is centered on building high performance, scalable data pipelines and processing large datasets in a distributed environment. You will work primarily on Spark based data processing running on Azure Databricks, developing production grade code that supports enterprise analytics, reporting, and data products.

Responsibilities

- Design, develop, and maintain large‑scale Spark applications using Python, Scala, and/or Java
- Build and optimize batch and streaming data pipelines in distributed environments
- Write production‑quality Spark code with strong focus on performance, scalability, and reliability
- Optimize Spark jobs (partitioning, caching, shuffles, memory tuning, execution plans)
- Develop reusable Spark frameworks, libraries, and utilities
- Work with structured and semi‑structured data (Parquet, Delta, CSV, JSON)
- Collaborate with platform, analytics, and data science teams to support downstream use cases
- Debug and troubleshoot Spark job failures and performance issues in production
- Follow best practices for code quality, testing, logging, and documentation

Qualifications

- Bachelor’s degree in Computer Science, Engineering, or a related field (or equivalent experience)
- 3+ years of experience as a Data Engineer or in a similar role
- Strong hands‑on experience with Apache Spark
- Proficiency in one or more of the following languages: Python, Scala or Java




- Strong understanding of distributed systems and data processing concepts
- Solid experience with SQL for data transformation and analysis
- Experience working with data lakes and columnar formats (e.g., Parquet, Delta Lake)
- Familiarity with Git and standard software engineering practices
- Strong problem‑solving skills and attention to detail
- Experience running Spark on Azure Databricks (asset)
- Experience migrating Spark jobs from on‑prem Hadoop/Cloudera to cloud platforms (asset)
- Familiarity with orchestration tools (e.g., Airflow, Azure Data Factory) (asset)
- Exposure to cloud storage (ADLS Gen2) and cloud security concepts (asset)

Benefits

- Diversity, Equity, Inclusion & Allyship — inclusive culture, Employee Resource Groups, and bias‑free practices
- Accessibility and Workplace Accommodations — ongoing commitment to inclusive and accessible environment
- Upskilling through online courses, cross‑functional development opportunities, and tuition assistance
- Competitive Rewards program including bonus, flexible vacation, personal, sick days and benefits starting on day one
- Dynamic Ecosystem — free tea & coffee, universal washrooms, and space for team collaboration
- Community Engagement — program opportunities for engagement and belonging

Location(s): Canada: Ontario : Toronto

If you require accommodation (including, but not limited to, an accessible interview site, alternate format documents, ASL Interpreter, or Assistive Technology) during the recruitment and selection process, please let our Recruitment team know.

#J-18808-Ljbffr

Apply on Kit Job: kitjob.ca/job/2ghksh

📌 Data Engineer (Toronto)
🏢 Scotiabank
📍 Toronto