22 Nov
|
Blutic
|
Montreal
Blutic is looking for an experienced Data Engineer with 5+ years of experience and strong hands-on expertise in designing and building scalable ETL/ELT pipelines using Azure Data Factory (ADF), Azure Databricks (PySpark), and Snowflake. This is W2 (Hybrid) chance.
Location:
Montreal, Canada
Experience:
5+ Years Required
Qualifications:
• 5+ years in data engineering or ETL development
• Strong hands-on experience with Snowflake (Snowpipe, Streams and Tasks, performance tuning).
• Hands-on Azure Databricks and PySpark, including Delta Lake.
• Proficiency in Azure Data Factory, including parameterized and metadata-driven pipelines.
• Strong SQL and PL/SQL or T-SQL, including stored procedures.
• Experience with Informatica PowerCenter or IICS, ideally including migration to cloud platforms.
• Python for data processing, validation, and automation.
• Experience with data validation and reconciliation in financial or regulated environments.
• Familiarity with Git, Azure DevOps CI/CD, and Agile/Scrum delivery.
Key Responsibilities
• Design and build scalable ETL/ELT pipelines using Azure Data Factory, Azure Databricks (PySpark), and Snowflake.
• Develop Medallion architecture (Bronze, Silver, Gold) on Delta Lake, using Delta Live Tables and Unity Catalog for governance.
• Build Snowflake solutions using Snowpipe, Streams, Tasks, Dynamic Tables, and Snowpark (Python).
• Create metadata-driven, parameterized, reusable pipeline frameworks to speed up source onboarding.
• Write and tune complex SQL, PL/SQL, and T-SQL, including stored procedures, functions, and packages.
• Implement CDC and incremental loading (Delta Lake MERGE, watermark logic) across batch and near real-time pipelines.
• Build data quality, reconciliation, auditing, and exception-handling frameworks to support regulatory and financial reporting.
• Optimize performance and cost across Databricks clusters, Snowflake warehouses, and ADF pipelines.
• Automate CI/CD with Azure DevOps, Git, and Terraform; support release management across environments.
• Provide production support, root cause analysis, and monitoring for critical pipelines.
• Work with architects, analysts, finance, risk, and compliance teams, and mentor junior engineers.
📌 Data Engineer | Hybrid | Montreal, Canada
🏢 Blutic
📍 Montreal