Title: Senior Data Engineer (Databricks)

Location: Toronto, ON - Hybrid

Experience: 8+ years

Responsibilities:
Design, develop, and maintain scalable ETL/ELT pipelines for structured and unstructured data.
Build and optimize data solutions using Databricks, PySpark, and Spark SQL.
Implement data ingestion frameworks from various source systems including databases, APIs, files, and streaming platforms.
Develop and maintain data lakehouse architectures leveraging Databricks best practices.
Design and implement Medallion Architecture (Bronze, Silver, Gold layers) for enterprise data platforms.
Collaborate with business stakeholders, data scientists, architects, and application teams to understand data requirements.
Optimize Spark jobs and data pipelines for performance, scalability, and cost efficiency.
Implement data quality, governance, security, and monitoring frameworks.
Support AI/ML initiatives by preparing and engineering datasets for model development and deployment.
Develop CI/CD pipelines and automate deployment processes for data engineering workloads.
Participate in architecture reviews and establish data engineering standards and best practices.
Mentor junior engineers and provide technical leadership across projects.

Required Qualifications




Bachelor's or Master's degree in Computer Science, Information Technology, Data Science, or related field.
8+ years of experience in Data Engineering and Data Platform development.
Solid hands-on experience with Databricks and Apache Spark.
Proficiency in Python, PySpark, and SQL.
Experience with cloud platforms such as:

o Microsoft Azure (preferred)

o AWS o Google Cloud Platform
Experience with Delta Lake, Unity Catalog, and Databricks workflows.
Robust understanding of Data Lake, Data Warehouse, and Lakehouse architectures.
Experience working with large-scale distributed data processing systems.
Knowledge of data modeling, partitioning, indexing, and query optimization techniques.
Experience with Git, CI/CD pipelines, and DevOps practices.

Preferred Qualifications
Databricks Certified Data Engineer Associate or Skilled Certification.
Experience with Azure Data Factory, Synapse Analytics, or equivalent cloud-native services.
Experience with streaming technologies such as Kafka, Event Hubs, or Spark Streaming.
Exposure to AI/ML platforms, MLOps, and Generative AI solutions.
Experience working in Insurance, Financial Services, or Enterprise Digital Transformation programs.

📌 Senior Data Engineer Databricks Toronto
🏢 Yochana
📍 Toronto