Responsibilities

- Lead the architecture, design, and implementation of advanced analytics solutions using Azure Databricks and Microsoft Fabric.

- Apply deep expertise in big data technologies, data engineering, cloud computing, Azure Databricks, and solid SQL skills to deliver scalable solutions.

- Work closely with business stakeholders and IT teams to understand requirements and deliver effective, business-aligned solutions.

- Oversee end-to-end implementation of data solutions, ensuring alignment with business requirements and industry best practices.

- Lead the development of data pipelines and ETL processes using Azure Databricks, PySpark, and other relevant tools.

- Integrate Azure Databricks with Azure services such as Azure Data Lake, Azure Synapse, Azure Data Factory, and on-premises systems.

- Provide expertise in Spark; Scala experience is an added advantage.

- Demonstrate strong Microsoft Fabric expertise, including pipeline development, notebook development, semantic modelling, real-time intelligence, and related scenarios.

- Provide technical leadership and mentorship to the data engineering team, fostering a culture of continuous learning and improvement.

- Ensure proper documentation of architecture, processes, and data flows while maintaining compliance with security and governance standards.

- Promote best practices for code quality, data security, scalability, and maintainability.

- Stay current with the latest developments in Databricks and associated technologies to support continuous innovation.

Essential Skills

- Mandatory French-speaking capability.

- Strong experience with Azure Databricks, including cluster management, notebook development, and Delta Lake.

- Proficiency in big data technologies such as Hadoop and Spark, along with data processing frameworks such as PySpark.

- Deep understanding of Azure services,



including Azure Data Lake, Azure Synapse, and Azure Data Factory.

- Experience with Power BI, SSRS, SSAS, and SKU migration from P-SKU to F-SKU.

- Experience with ETL/ELT processes, data warehousing, and data lake development.

- Strong SQL skills and familiarity with NoSQL databases.

- Experience with CI/CD pipelines and version control systems such as Git.

- Knowledge of cloud security best practices.

Requirements

Soft Skills

- Mandatory French-speaking capability.

- Excellent communication skills, with the ability to explain complex technical concepts to non-technical stakeholders.

- Strong problem-solving skills and a proactive approach to identifying and resolving issues.

- Leadership skills, with the ability to manage and mentor a team of data engineers.

Nice-to-Have Skills

- Power BI experience for dashboarding and reporting.

- Microsoft Fabric experience for analytics and integration tasks.

- Spark Streaming experience for processing real-time data streams.

- Familiarity with Azure Resource Manager templates for infrastructure as code practices.

Experience

- Demonstrated expertise of 12 years in developing data ingestion and transformation pipelines using Databricks/Synapse notebooks and Azure Data Factory.

- Solid understanding and hands-on experience with Delta tables, Delta Lake, and Azure Data Lake Storage Gen2.

- Experience using Auto Loader and Delta Live Tables for seamless data ingestion and transformation.

- Proficiency in building and optimizing query layers using Databricks SQL.

- Demonstrated experience integrating Databricks with Azure Synapse, ADLS Gen2, and Power BI for end-to-end analytics solutions.

- Prior experience developing, optimizing, and deploying Power BI reports.

- Familiarity with modern CI/CD practices, especially in the context of Databricks and cloud-native solutions.

📌 Specialist - Architecture (East York)
🏢 CareerBeacon
📍 East York