Key Responsibilities

- Design, develop, and maintain scalable ETL/ELT pipelines using Azure Data Factory (ADF).
- Develop data transformation and processing solutions using Azure Databricks and PySpark.
- Build and manage data storage solutions using Azure Data Lake Storage (ADLS Gen2).
- Write complex SQL queries, stored procedures, and optimize query performance.
- Implement incremental data loads, CDC, and batch processing.
- Work with Delta Lake for data management, including MERGE, UPDATE, and DELETE operations.
- Develop data pipelines following Medallion Architecture (Bronze, Silver, Gold).
- Implement data integration from multiple sources such as databases, APIs, and flat files.
- Monitor, troubleshoot, and optimize data pipelines for performance and reliability.
- Implement security using Azure Key Vault, Managed Identity, and RBAC.
- Work on CI/CD deployments using Azure DevOps and Git.




- Collaborate with business analysts, data architects, and other technical teams. Preferred Skills
- Hands-on experience with Azure Data Factory (ADF).
- Solid knowledge of Azure Databricks and PySpark.
- Proficiency in Python programming and SQL.
- Experience with Azure Data Lake Storage Gen2.
- Knowledge of Azure Synapse Analytics.
- Experience with Delta Lake and Lakehouse Architecture.
- Understanding of ETL/ELT concepts and data warehousing.
- Experience with incremental loading and Change Data Capture (CDC).
- Knowledge of Azure DevOps, Git, and CI/CD pipelines.
- Understanding of data modeling and data quality concepts.
- Good problem-solving and analytical skills.
- Excellent communication and collaboration skills.

📌 Azure Data Engineer (Canada)
🏢 Spait Infotech
📍 Canada