Required Skills & Qualifications

- Degree or diploma in Computer Science, Software Engineering, Information Technology, or a related field.

- Hands-on experience developing data engineering solutions using

Azure Data Factory (ADF), Azure Data Lake Storage (ADLS), and Azure Databricks

.

- Strong programming skills in
Python, SQL, PySpark, and Apache Spark for data processing and transformation.

- Experience developing and maintaining

ETL/ELT pipelines

, data ingestion frameworks, and scalable batch or near real-time data processing solutions.

- Hands-on knowledge of

Delta Lake, Lakehouse Architecture, and Medallion Architecture (Bronze, Silver, Gold)

.

- Experience developing Databricks notebooks, workflows, data processing jobs, and optimizing Delta Lake tables.

- Strong understanding of data validation, data quality, reconciliation, monitoring,



and troubleshooting of data pipelines.

- Experience with
Azure DevOps, Git, CI/CD pipelines, automated deployments, and version control

.

- Knowledge of
REST APIs, data integration patterns, automation frameworks, and reusable development utilities

.

- Understanding of data governance, metadata management, data cataloguing, security, privacy, and access control

.

- Experience with software development best practices, including code reviews, automated testing, documentation, and secure coding.

- Robust communication and collaboration skills, with experience working in

Agile/Scrum teams alongside architects, data engineers, analysts, and business stakeholders.

📌 Junior Data Engineer – Azure Databricks / ADF / PySpark (Toronto)
🏢 ThoughtStorm
📍 Toronto