Team Lead, Data Engineering - Databricks to lead and mentor a team of data engineers, conducting code reviews, design reviews, and knowledge-sharing sessions across multiple locations

8+ years of experience in

data engineering

3+ years of hands‑on experience with

Databricks

platform

Proven experience

leading a team

Strong expertise in

Python and Spark

programming

Demonstrable experience in using

AI

in development

Proven experience with

AWS

or other similar cloud services

Deep understanding of data modeling and SQL

Experience with

Delta Lake and Lakehouse

architecture

Strong knowledge of

ETL/ELT

principles and patterns

Experience with version control systems (Git)

Demonstrated ability to optimize data pipelines

Strong problem-solving and analytical skills

Excellent communication and collaboration abilities

Nice to Have:

Financial services industry experience

Experience with multiple cloud providers

Knowledge of AI/ML implementation patterns

Experience with real time data processingData governance framework experience

What Sets You Apart:

Active interest in emerging technologies and industry trends

Experience implementing complex data solutions

Track record of continuous learning and skill development

Strong technical documentation abilities

Experience using AI tools to enhance productivity

Ability to mentor junior team members

Technical Environment:

Primary Platform: Databricks

Languages: Python, SQL

Tools: Delta Lake, Unity Catalog, Git

Additional: Real-time processing, API integrations

Responsibilities

Your Role:

You will lead and mentor a team of data engineers, conducting code reviews, design reviews, and knowledge-sharing sessions across multiple locations

Drive the Agile/Scrum SDLC process and collaborate with team members





Design and develop Databricks solutions leveraging Lakehouse architecture for enterprise data processing and analytics

Develop and optimize ETL/ELT pipelines

Create and manage structured streaming pipelines for real-time data processing

Configure and optimize Databricks clusters and Spark jobs for optimal performance

Utilize Delta Live Tables for data ingestion and transformations

Apply Unity Catalog features and IAM best practices for security governance and access control

Support infrastructure and resource management using Terraform

Implement monitoring solutions for pipeline performance and data quality

Contribute to code reviews and knowledge-sharing sessions

Salary Range: CAD $135,000 - CAD $175,000

Permanent Full Time

Hybrid (3 days in the office)

Must Haves:

8+ years of experience in

data engineering

3+ years of hands‑on experience with

Databricks

platform

Proven experience

leading a team

Robust expertise in

Python and Spark

programming

Demonstrable experience in using

AI

in development

Proven experience with

AWS

or other similar cloud services

Deep understanding of data modeling and SQL

Experience with

Delta Lake and Lakehouse

architecture

Strong knowledge of

ETL/ELT

principles and patterns

Experience with version control systems (Git)

Demonstrated ability to optimize data pipelines

Strong problem-solving and analytical skills

Excellent communication and collaboration abilities

Nice to Have:

Financial services industry experience





Experience with multiple cloud providers

Knowledge of AI/ML implementation patterns

API development experience

Experience with real time data processing

Data governance framework experience

What Sets You Apart:

Active interest in emerging technologies and industry trends

Experience implementing complex data solutions

Track record of continuous learning and skill development

Strong technical documentation abilities

Experience using AI tools to enhance productivity

Ability to mentor junior team members

Technical Environment:

Primary Platform: Databricks

Cloud Platform: AWS (S3, Glue, Lambda)

Languages: Python, SQL

Tools: Delta Lake, Unity Catalog, Git

Additional: Real-time processing, API integrations

Responsibilities

Your Role:

You will lead and mentor a team of data engineers, conducting code reviews, design reviews, and knowledge-sharing sessions across multiple locations

Drive the Agile/Scrum SDLC process and collaborate with team members

Design and develop Databricks solutions leveraging Lakehouse architecture for enterprise data processing and analytics

Develop and optimize ETL/ELT pipelines

Create and manage structured streaming pipelines for real-time data processing

Configure and optimize Databricks clusters and Spark jobs for optimal performance

Utilize Delta Live Tables for data ingestion and transformations

Apply Unity Catalog features and IAM best practices for security governance and access control

Support infrastructure and resource management using Terraform

Implement monitoring solutions for pipeline performance and data quality

Contribute to code reviews and knowledge-sharing sessions

Salary Range: CAD $135,000 - CAD $175,000

Disclaimer: AI may be used in evaluating candidates. This posting is for an existing vacancy.

#J-18808-Ljbffr

📌 Team Lead, Data Engineering - Databricks to lead and mentor a team of data engineers, conducting cod (Winnipeg)
🏢 S.I. Systems
📍 Winnipeg