Key Responsibilities

- Design and develop scalable data pipelines using Azure services.

- Build and maintain ETL/ELT workflows for structured and unstructured data.

- Develop data integration solutions using

Azure Data Factory (ADF)

.

- Store and process data using

Azure Data Lake Storage (ADLS)

.

- Develop and optimize data transformations using

Azure Databricks and PySpark.

- Design and maintain data warehouses using

Azure Synapse Analytics

.

- Write optimized SQL queries, stored procedures, and data transformations.

- Perform data cleansing, validation, and quality checks.

- Integrate data from multiple sources such as databases, APIs, files, and cloud applications.

- Monitor pipeline performance and troubleshoot data-processing failures.

- Implement data security, access control, and governance practices.

- Optimize data pipelines and cloud resources for performance and cost.

- Collaborate with data analysts, data scientists, software engineers, and business stakeholders.

- Maintain technical documentation for data pipelines, data models, and architecture.

Required Technical Skills

Azure

- Azure Data Factory (ADF)

- Azure Data Lake Storage Gen2

- Azure Databricks

- Azure Synapse Analytics





- Azure Key Vault

- Azure Monitor

- Azure Storage

- Azure SQL Database

Programming & Data

- Solid

SQL

- Python

- PySpark

- ETL/ELT concepts

- Data transformation and data cleansing

- Data modeling

- Data warehousing concepts

Database Technologies

- SQL Server

- Azure SQL

- PostgreSQL/MySQL

- Relational and dimensional databases

Data Engineering Concepts

- ETL/ELT pipeline development

- Batch and incremental data processing

- Star and snowflake schemas

- Fact and dimension tables

- Slowly Changing Dimensions (SCD)

- Data partitioning

- Data quality and validation

- Performance optimization

- Data lake and data warehouse architecture

DevOps & Version Control

- Git/GitHub/Azure Repos

- CI/CD pipelines

- Azure DevOps

- Infrastructure and deployment automation

- Environment management across development, testing, and production

Preferred Skills

- Microsoft Fabric

- Azure Event Hubs

- Azure Stream Analytics

- Azure Functions

- Power BI integration

- Delta Lake

- Apache Spark

- Kafka

- Terraform

- Cloud security and governance

- Microsoft Purview

📌 Azure Data Engineer (Canada)
🏢 Spait Infotech
📍 Canada