Lead Databricks migrations with Python, PySpark, and SQL from a hybrid Toronto engagement. Modernize production pipelines for two-week Agile delivery.

This senior data engineering contract spans two months and focuses on migrating existing ETL/ELT workloads into Databricks. You will refactor processing logic, build scalable pipelines, and maintain ingestion, transformation, and metric calculations. The work includes Airflow orchestration, source-to-target mappings, testing, reconciliation, and production transition activities.

Key Responsibilities

Migrate and refactor ETL/ELT pipelines into Databricks

Develop production pipelines using Python, PySpark, and SQL

Modernize scheduled jobs and Airflow DAG-based processing





Implement reusable patterns for additional workload migrations

Support deployment readiness and production-transition activities

Requirements

Seven or more years in data or software engineering

Solid hands-on Databricks production experience

Advanced Python, PySpark, Apache Spark, and SQL skills

Experience migrating legacy pipelines to modern cloud platforms

Greater Toronto Area residence and downtown onsite availability

Apply senior Databricks engineering expertise to modernize pipelines through Python, PySpark, SQL, orchestration, and migration delivery.

📌 Senior Databricks Migration Engineer (Winnipeg)
🏢 Ian Martin Group
📍 Winnipeg