Data Engineer / Software Developer

Onsite - Toronto, ON

6 Months

Key Responsibilities

Data Engineering & Pipeline Development

- Design, develop, and maintain scalable batch and near real-time data pipelines using Azure Data Factory (ADF), Databricks, and related cloud technologies.
- Develop data ingestion frameworks to onboard data from various source systems into the Ontario Data Platform.
- Build and optimize ETL/ELT processes to transform, cleanse, and prepare data for analytics and reporting.
- Implement data validation, monitoring, and reconciliation processes to ensure data quality and reliability.

Databricks Development

- Develop and maintain notebooks, workflows, and data processing jobs using Databricks.
- Build data transformation pipelines leveraging Spark and Databricks Lakehouse architecture.
- Support the implementation of medallion architecture patterns (Bronze, Silver, Gold).
- Monitor and troubleshoot Databricks jobs, workflows, and performance issues.
- Develop and manage Delta Lake tables and optimize data processing workloads for performance and scalability.

Data Storage & Integration

- Design and manage data solutions utilizing Azure Data Lake Storage (ADLS).
- Collaborate with source system teams to establish secure and productive data integration patterns.
- Implement data partitioning,



optimization, and archival strategies to improve performance and scalability.
- Support metadata management, data cataloguing, and governance initiatives.

Software Development & Automation

- Develop reusable frameworks, scripts, and utilities to automate data engineering processes.
- Participate in DevOps practices, including CI/CD pipelines, code reviews, testing, and deployment automation.
- Maintain technical documentation, design specifications, and operational procedures.
- Contribute to platform enhancements and continuous improvement initiatives.
- Apply software engineering best practices including version control, test automation, code quality standards, and secure coding principles.

Collaboration & Stakeholder Engagement

- Work closely with architects, product managers, data scientists, analysts, and business stakeholders to understand requirements and deliver technical solutions.
- Participate in Agile teams and contribute to sprint planning, estimation, and delivery activities.
- Provide technical guidance and knowledge sharing to team members and partners.
- Participate in solution architecture reviews and contribute to technology roadmaps and design decisions.

📌 Data Engineer / ETL Developer (Toronto)
🏢 I8IS
📍 Toronto