19 Aug
|
Tekshapers
|
Toronto
19 Aug
Tekshapers
Toronto
Job Title – Azure DataBrick
Job Location – Toronto (Hybrid) Canada
Duration – 6+ Months
Experience level – 6+ Years
Mandatory Skills –Azure Databrick , ETL Pipeline, Delta Lake, SQL, CI/CD
Required Skills & Experience:
Design and develop ETL pipelines using Azure Databricks and Delta Lake, focusing on batch processing (autoloader) and Spark structured streaming.
Create and manage end-to-end environments, including catalogs, schemas, tables, materialized views, functions, and volumes using Unity Catalog.
Implement slowly changing dimensions (SCD1 and SCD2) on dimension tables and build change data capture (CDC) pipelines.
Utilize Lakehouse federation to create foreign catalogs for accessing data from external sources.
Optimize data processing through effective partitioning and liquid clustering in Databricks.
Collaborate with cross-functional teams to ensure data governance and security practices are adhered to.
Participate in CI/CD pipeline development and DevOps practices to enhance deployment efficiency.
Mandatory Skills
Expertise in Azure Databricks
Expert-level proficiency in SQL
Regular experience with CI/CD practices
Tekshapers is an equal prospect employer and will consider all applications without regards to race, sex, age, color, religion, national origin, veteran status, disability, sexual orientation, gender identity, genetic information or any characteristic protected by law.
📌 Azure DataBrick (Toronto)
🏢 Tekshapers
📍 Toronto