08 Nov
|
Softline Technology
|
Toronto
08 Nov
Softline Technology
Toronto
Description
Role Overview
The Government of Ontario is undertaking a large-scale Data & AI transformation initiative, with the Ontario Data Platform (ODP) serving as the enterprise technology ecosystem for data-driven decision-making, analytics, and artificial intelligence. ODP provides a modern and scalable data environment supporting data ingestion, storage, processing, governance, quality, marketplace capabilities, and advanced analytics across ministries and government organizations.
We are seeking a Data Engineer / Software Developer to support the design, development, and operation of enterprise data solutions on the Ontario Data Platform. The successful candidate will work with up-to-date cloud technologies, including Databricks, Azure Data Factory (ADF), and Azure Data Lake Storage (ADLS), to build scalable data pipelines, enable analytics use cases, and support the delivery of high-quality data products across government.
Experience Requirements
Key Responsibilities
Data Engineering & Pipeline Development
- Design, develop, and maintain scalable batch and near real-time data pipelines using Azure Data Factory (ADF), Databricks, and related cloud technologies.
- Develop data ingestion frameworks to onboard data from various source systems into the Ontario Data Platform.
- Build and optimize ETL/ELT processes to transform, cleanse, and prepare data for analytics and reporting.
- Implement data validation, monitoring, and reconciliation processes to ensure data quality and reliability.
Databricks Development
- Develop and maintain notebooks, workflows,
and data processing jobs using Databricks.
- Build data transformation pipelines leveraging Spark and Databricks Lakehouse architecture.
- Support the implementation of medallion architecture patterns (Bronze, Silver, Gold).
- Monitor and troubleshoot Databricks jobs, workflows, and performance issues.
- Develop and manage Delta Lake tables and optimize data processing workloads for performance and scalability.
Data Storage & Integration
- Design and manage data solutions utilizing Azure Data Lake Storage (ADLS).
- Collaborate with source system teams to establish secure and productive data integration patterns.
- Implement data partitioning, optimization, and archival strategies to improve performance and scalability.
- Support metadata management, data cataloguing, and governance initiatives.
Software Development & Automation
- Develop reusable frameworks, scripts, and utilities to automate data engineering processes.
- Participate in DevOps practices, including CI/CD pipelines, code reviews, testing, and deployment automation.
- Maintain technical documentation, design specifications, and operational procedures.
- Contribute to platform enhancements and continuous improvement initiatives.
- Apply software engineering best practices including version control, test automation, code quality standards, and secure coding principles.
Collaboration & Stakeholder Engagement
- Work closely with architects, product managers, data scientists, analysts, and business stakeholders to understand requirements and deliver technical solutions.
- Participate in Agile teams and contribute to sprint planning, estimation, and delivery activities.
- Provide technical guidance and knowledge sharing to team members and partners.
- Participate in solution architecture reviews and contribute to technology roadmaps and design decisions.
Evaluation breakdown: Technical(Power BI) – 80%;
Communication – 10%;
Public Sector exp– 10%
Supplier Comments
Closing Date/Time: 2026-11-10, 12:00 p.m. EST.
Max submission: 1 (one)
5 days onsite
Qualifications
- Degree or diploma in Computer Science, Software Engineering, Information Technology, or a related field.
- Experience developing data solutions using Azure Data Factory (ADF), Azure Data Lake Storage (ADLS), Databricks, Delta Lake and Lakehouse architecture principles.
- Experience working with Azure DevOps, Git, CI/CD pipelines, and automated deployment practices.
- Strong proficiency in Python, SQL, and Spark-based data processing frameworks.
- Knowledge of data governance, security, privacy, and access control principles.
- Experience with REST APIs, application integration patterns, and automation frameworks.
📌 Report Developer (Toronto)
🏢 Softline Technology
📍 Toronto