Senior Data Engineer –Databricks 3 months Contract with possibility of extension. Work Arrangement
- Remote Please find below the details JD:

Key Responsibilities
• Develop and implement data solutions using Databricks, with hands-on experience in specific Databricks platform features such as Delta Lake, Uniform (Iceberg), Delta Live Tables (Lakeflow Declarative Pipelines), and Unity Catalog.
• Collaborate with cross-functional teams to design and optimize data pipelines for both batch and streaming data, ensuring data quality and efficiency.
• Stay updated with emerging technologies, latest Databricks platform features and continuously improve your skills in Databricks and other relevant tools. Mandatory

Requirements:
• 5+ years of data engineering experience, with at least 2 years of hands-on data pipeline design and development experience with Databricks, including specific -platform features like Delta Lake, Uniform (Iceberg), Delta Live Tables (Lakeflow Declarative pipelines), and Unity Catalog.
• Proficiency in designing and building robust, scalable, YAML configuration driven data pipelines with batch, micro-batch, and streaming data ingestion and processing patterns using tools like Auto Loader demonstrating a solid understanding of contemporary data engineering practices.




Experienced in building complex job/workflow orchestration patterns using either Databricks jobs/workflows or with other orchestration tools like Airflow, dbt etc.
• Exposure in building robust data quality, pipeline audit/observability solutions with
• Databricks native features and/or other data quality tools/frameworks like Great Expectations, Collibra, dbt etc.
• Proficiency in Big Data Platforms: Apache Spark, Presto, Amazon EMR.
• Experience with Cloud Data Warehouses: Amazon Redshift, Snowflake, Google BigQuery.
• Strong programming skills using SQL, Stored Procedures and Object-Oriented Programming languages (like Java, Python, PySpark etc.).
• Familiarity with building DevOPs, CI/CD pipelines using Databricks Asset Bundles with automated validation and testing.
• Exposure with Infrastructure as Code (IaC) tools like Terraform is a big plus.
• Familiarity with NoSQL Databases and Container Management Systems.
• Exposure to AI/ML tools (like mlflow), prompt engineering, and modern data and AI agentic workflows.
• An ideal candidate will have Databricks Data Engineering Associate and/or Skilled certifications completed with multiple Databricks project delivery experience.

📌 Senior Data Engineer –Databricks (Canada)
🏢 Ideaon
📍 Canada