Apply on Kit Job: kitjob.ca/job/2fj2cy

Permanent Full Time

Hybrid (3 days in the office)

Must Haves:

- 8+ years of experience in data engineering
- 3+ years of hands-on experience with Databricks platform
- Proven experience leading a team
- Strong expertise in Python and Spark programming
- Demonstrable experience in using AI in development
- Proven experience with AWS or other similar cloud services
- Deep understanding of data modeling and SQL
- Experience with Delta Lake and Lakehouse architecture
- Strong knowledge of ETL/ELT principles and patterns
- Experience with version control systems (Git)
- Demonstrated ability to optimize data pipelines
- Robust problem-solving and analytical skills
- Excellent communication and collaboration abilities

Nice to Have:

- Financial services industry experience
- Experience with multiple cloud providers
- Knowledge of AI/ML implementation patterns
- API development experience
- Experience with real-time data processing
- Data governance framework experience

What Sets You Apart:

- Active interest in emerging technologies and industry trends
- Experience implementing complex data solutions
- Track record of continuous learning and skill development
- Strong technical documentation abilities
- Experience using AI tools to enhance productivity
- Ability to mentor junior team members

Technical Environment:

- Primary Platform: Databricks




- Cloud Platform: AWS (S3, Glue, Lambda)
- Languages: Python, SQL
- Tools: Delta Lake, Unity Catalog, Git
- Additional: Real-time processing, API integrations

Responsibilities

Your Role:

- You will lead and mentor a team of data engineers, conducting code reviews, design reviews, and knowledge-sharing sessions across multiple locations
- Drive the Agile/Scrum SDLC process and collaborate with team members
- Design and develop Databricks solutions leveraging Lakehouse architecture for enterprise data processing and analytics
- Develop and optimize ETL/ELT pipelines
- Create and manage structured streaming pipelines for real-time data processing
- Configure and optimize Databricks clusters and Spark jobs for optimal performance
- Utilize Delta Live Tables for data ingestion and transformations
- Apply Unity Catalog features and IAM best practices for security governance and access control
- Support infrastructure and resource management using Terraform
- Implement monitoring solutions for pipeline performance and data quality
- Contribute to code reviews and knowledge-sharing sessions

Salary Range: CAD $135,000 - CAD $175,000

Disclaimer:
AI may be used in evaluating candidates.
This posting is for an existing vacancy.

Apply

Apply on Kit Job: kitjob.ca/job/2fj2cy

📌 Team Lead, Data Engineering - Databricks to lead and mentor a team of data engineers, conducting code reviews, design reviews, and knowledge-sharing sessio (Toronto)
🏢 S.i. Systems
📍 Toronto