23 Oct
|
vTech Solution
|
Ontario
23 Oct
vTech Solution
Ontario
Job Summary: The Senior Data
Ops/Cloud Data Engineer is responsible for designing, developing, and optimizing scalable data pipelines and cloud-based data platforms.
This role focuses on implementing Data
Ops best practices, automating workflows, and supporting enterprise data solutions including data lakes, warehouses, and lakehouses in cloud environments.
The engineer will collaborate with cross-functional teams to enable data integration, quality, governance, and migration initiatives.
Responsibilities: Design and develop end-to-end data pipelines from source systems to end users.
Optimize and automate data workflows and pipeline performance.
Implement Data
Ops practices including CI/CD, deployment automation, and monitoring.
Develop and maintain cloud data platforms and services including data lakes, warehouses, and lakehouses.
Automate metadata collection, data quality, and lineage enablement.
Support cloud modernization and data migration projects.
Collaborate with architects, data modelers, and governance teams to operationalize data standards and metadata.
Troubleshoot complex data engineering and integration issues.
Ensure data security, governance, and operational best practices are followed.
Required Skills: Proven experience with Databricks, Delta Lake, and Databricks SQL.
Robust knowledge of Medallion Architecture.
Experience with Azure Data Services and Azure Data Factory.
Proficiency with Informatica or equivalent data integration platforms.
Operationalizing common data models and metadata repositories.
Expertise in designing and supporting cloud-native data platforms and scalable ETL/ELT pipelines.
Experience implementing Data
Ops including deployment automation,
testing, and operational support.
Strong analytical, problem-solving, and communication skills.
Desired Skills: Experience with Microsoft Fabric and Microsoft Purview.
Knowledge of graph technologies and semantic repositories.
Familiarity with AODA and related accessibility standards.
Experience working in public sector or operational environments.
Ability to adapt to tight deadlines and shifting project priorities.
Rated Criteria: Cloud Data Platform Design and Implementation - 25 Points Data Pipeline Development and Optimization - 20 Points Data
Ops and CI/CD Automation - 15 Points Metadata, Data Quality, and Lineage Enablement - 15 Points Experience with Databricks and Delta Lake Technologies - 15 Points Integration and Migration Expertise - 10 Points Total: 100 Points Deliverables: Robust data pipelines and integration services supporting enterprise data needs.
Automated metadata collection and synchronization processes.
Cloud data platform implementation and configuration documentation.
Enabled data quality and lineage capabilities within cloud environments. CI/CD pipelines and Data
Ops automation frameworks.
Data migration and interoperability services to cloud platforms.
Cloud-hosted reusable data products and services.
Metadata and governance platform integrations supporting enterprise standards.
Must Haves: Experience with Databricks, Delta Lake, and Databricks SQL.
Knowledge of Medallion Architecture.
Hands-on experience with Azure Data Services and Azure Data Factory.
Proficiency with Informatica or equivalent integration tools.
Ability to operationalize common data models and metadata repositories.
Public Sector Experience: Preferred
📌 Senior Cloud Data Engineer (Ontario)
🏢 vTech Solution
📍 Ontario