29 Aug
|
Xinova Group
|
Toronto
29 Aug
Xinova Group
Toronto
We are looking for a Senior Data Engineer to join a fast-paced, data-driven setting on a contract basis. This role is ideal for an engineer with deep hands-on expertise across the Azure and Databricks ecosystem who can design, build, and optimize enterprise-scale data platforms. You'll play a key role in modernizing ETL/ELT processes, strengthening data governance, and driving cost and performance efficiency across the Databricks Lakehouse.Key ResponsibilitiesDesign, develop, and maintain scalable ETL/ELT pipelines using Azure and Databricks to support enterprise data and analytics needsBuild and manage Delta Live Tables (DLT) pipelines for reliable, automated data transformation workflowsImplement and maintain Unity Catalog for centralized data governance, access control, and lineage tracking across the LakehouseDesign and develop semantic layers to support consistent, business-friendly access to curated data for analytics and BI consumptionBuild and maintain CI/CD pipelines for data engineering workflows, ensuring automated testing, deployment, and version control best practicesOwn FinOps practices within Databricks — monitoring, forecasting, and optimizing compute/storage costs across clusters and jobsConduct performance tuning and optimization of Databricks workloads, including cluster configuration, query optimization, and job orchestrationWork across core Azure data services (e.G., Azure Data Factory, Azure Data Lake Storage, Azure Synapse) to integrate and orchestrate data pipelinesPartner with data architects, analysts, and business stakeholders to translate requirements into robust,
scalable data solutionsEnsure data quality, security, and compliance standards are upheld across all pipelines and platformsTroubleshoot and resolve pipeline failures, performance bottlenecks, and data quality issues in a timely mannerRequired Skills & ExperienceProven experience as a Senior Data Engineer, with a strong track record delivering production-grade data pipelinesStrong hands-on experience with Microsoft Azure (Azure Data Factory, ADLS, Azure Synapse, or similar)Deep expertise in Databricks, including:Unity CatalogCluster and job performance optimizationFinOps / cost management within DatabricksStrong experience building and maintaining CI/CD pipelines for data engineering workflows (e.G., Azure DevOps, GitHub Actions)Experience designing and implementing semantic layers for analytics/BI consumptionSolid background in ETL/ELT pipeline design, development, and optimizationProficiency in Python and/or SQL for data transformation and pipeline developmentStrong understanding of data governance, security, and lineage best practicesExcellent problem-solving skills and ability to work independently in a contract settingStrong communication skills, with the ability to work cross-functionally with technical and business stakeholdersNice to HaveDatabricks certifications (e.G., Databricks Certified Data Engineer Professional)Experience with Terraform or other infrastructure-as-code tools in an Azure/Databricks contextExperience in a regulated or enterprise-scale industryFamiliarity with Spark internals and advanced performance tuning#J-18808-Ljbffr
📌 Senior Data Engineer (Toronto)
🏢 Xinova Group
📍 Toronto