16 Nov
|
Celebal Technologies
|
Ottawa
16 Nov
Celebal Technologies
Ottawa
Our client is seeking an experienced, hands-on Senior Data Engineer to help build and scale the enterprise data foundation supporting Analytics, AI, and self-service decision-making across the company. This role requires more than just moving data between systems. The successful candidate will act as a data product architect, transforming complex, messy operational data into trusted, reusable, business-ready data products.
These products will power enterprise analytics, Tableau reporting, Databricks Genie / Genie Code , machine learning, and emerging GenAI use cases across Manufacturing, Supply Chain, Sales, Finance, and HR. You will play a pivotal role in shifting the organization away from one-off, siloed pipelines toward a highly governed, centralized single-source-of-truth platform built entirely on Databricks and AWS . Data Engineering & Lakehouse Development Design, develop, test, and maintain scalable ETL/ELT pipelines using Databricks, Spark, SQL, Python, and AWS .
Integrate structured and semi-structured data from enterprise applications, manufacturing systems, databases, APIs, and cloud operational sources.
Optimize
Databricks workloads, SQL queries, Delta tables, compute utilization, and end-to-end pipeline performance. Implement orchestration, monitoring, error handling, data reconciliation, and automated recovery for production data pipelines. Design scalable enterprise data models (dimensional modeling, star schemas, fact/dimension structures)
supporting cross-functional analytics and standardized KPIs.
Transform raw operational data into curated, business-ready data products that can be reused across dashboards, analytics, AI, and ML applications. Collaborate closely with business and analytics teams to translate complex business definitions into technically robust data models. Analytics AI & Self-Service Enablement Build AI-ready and Genie-ready data products that allow business users to securely analyze enterprise data using Databricks Genie and Genie Code.
Structure data, metadata, relationships, business definitions, and semantic/contextual layers to maximize the accuracy of conversational AI analytics. Enable engineers, analysts, and business users to perform governed self-service analysis without creating uncontrolled copies of data. Partner with data science teams to prepare feature-ready datasets for predictive analytics, machine learning, and MLOps workloads.
Data Governance, Quality & DataOps Implement data governance, cataloging, lineage, metadata management, and role-based access controls using Databricks Unity Catalog . Build automated data-quality checks for completeness, accuracy, consistency, freshness, and cross-system reconciliation.
Apply
Git-based development, code reviews, testing, version control, and automated CI/CD processes for Databricks assets. Focus heavily on developing reusable engineering frameworks and templates rather than building hard-coded, one-off pipelines. #
📌 Lead/Senior Data Engineer (Ottawa)
🏢 Celebal Technologies
📍 Ottawa