10 Sep
|
Systematix
|
Canada
We are Systematix and we are currently looking for an experienced Databricks Consultant – Data Architecture & Implementation for our client.
ABOUT THE PROJECT:
Our client is undertaking a financial reporting data modernization initiative as it transitions from a legacy claims platform to a current platform, Vision.
During this transition, Finance must continue reporting across both systems. More than 70 financial reports are currently built directly from Databricks Silver-layer datasets, with significant transformation and business logic embedded within individual reports to standardize and combine Legacy and Vision data. This has resulted in duplicated logic, inconsistent standardization, increased maintenance, and challenges around testing, reconciliation, traceability, governance and audit controls.
The client operates a modern Databricks Lakehouse environment hosted on Microsoft Azure, using a Medallion architecture and technologies including Delta Lake, Apache Spark, Unity Catalog, Gold-layer data marts, automated data-quality practices and CI/CD.
The objective of this engagement is to establish a scalable, governed and SOX-compliant financial reporting architecture, including evaluating whether a centralized Gold-layer financial data mart is the appropriate long-term solution.
The engagement will begin with a focused Phase 1 combining technical assessment with hands-on implementation. This is intended to be practical and time-bound rather than an extended strategy exercise. The majority of Phase 1 will focus on implementing the first usable solution for one priority financial reporting domain, or a small group of related reports, validating the architecture through production-ready deliverables before scaling further.
ABOUT THE RESPONSIBILITIES:
In this role, you will assess the current financial reporting architecture, recommend a practical target-state approach, and work hands-on to design and implement the initial solution. You will work closely with the Data Engineering Manager, as well as stakeholders across Finance, Analytics, Data Engineering and Internal Audit.
You will
- Assess the current Azure Databricks architecture, Silver-layer datasets, reporting processes and partially completed data mart work
- Review existing research, proposed materialized views, financial reporting requirements and SOX controls
- Identify technical, governance, scalability, maintainability and audit-related concerns
- Evaluate whether a centralized Gold-layer financial data mart is the appropriate long-term solution
- Design an approach for standardizing Legacy and Vision data and centralizing common financial reporting logic
- Design governed financial data models, dimensional models and reporting datasets
- Build or refactor Databricks pipelines, transformations, views, tables and Gold-layer data assets
- Implement appropriate reconciliation, validation, monitoring and data-quality controls
- Apply appropriate governance, lineage, access-control and documentation practices within the Databricks environment
- Establish or align with source-control, CI/CD, deployment and change-management practices
- Migrate agreed-upon priority financial reports to the new architecture
- Produce appropriate governance, audit and operational documentation
- Provide knowledge transfer to internal teams and recommendations for subsequent phases
Automated testing may also be recommended or implemented where practical and is considered a nice-to-have for Phase 1.
REQUIREMENTS:
Must-have:
- 7+ years of progressive data engineering and/or data architecture experience
- 4+ years of strong hands-on Databricks experience in enterprise environments
- Strong data architecture, data modelling and hands-on data engineering capabilities
- Experience designing and implementing Medallion architectures (Bronze/Silver/Gold)
- Experience designing and building Gold-layer data marts and governed reporting datasets
- Hands-on experience with Delta Lake and Apache Spark, including PySpark and/or SparkSQL
- Experience building and refactoring ETL/ELT pipelines and transformation logic
- Experience with Unity Catalog or comparable data governance, lineage and access-control capabilities
- Experience integrating and standardizing data from multiple source systems
- Experience implementing data reconciliation, validation, quality and monitoring controls
- Experience with source control and CI/CD practices in up-to-date data engineering environments
- Ability to move comfortably between architecture/design and hands-on implementation
- Strong written and verbal English communication skills with the ability to work effectively across technical and business stakeholders
Nice-to-have:
- Databricks certification, preferably Data Engineer Professional
- Relevant Microsoft Azure data certification
- Experience with Delta Live Tables (DLT) or similar orchestration capabilities
- Experience with Azure DevOps and/or GitHub Actions
- Experience with automated data-quality frameworks such as Databricks Expectations, Great Expectations or equivalent
- Financial reporting data and data-mart experience
- Experience working within SOX-controlled environments
- Experience supporting legacy-to-modern platform migrations
- Experience working directly with Finance and Internal Audit stakeholders
- Insurance, financial services or other regulated-industry experience
ABOUT THE ROLE:
Location: Remote – Canada (ET/PT)
Duration: 6-month contract / Start date: ASAP
Work hours: 37.5 hrs/week
Engagement: Contract with potential extension or contract-to-hire
Language: English
Reports to: Data Engineering Manager
PAY DISCLOSURE:
The average hourly pay range for this field is as follows:
- Junior: 2–4 years of relevant experience — $75–90/hr
- Intermediate: 5–7 years of relevant experience — $90–105/hr
- Senior: 8+ years of relevant experience — $105–120/hr
Compensation is commensurate with these standards; exceptions may apply based on experience, skills, and market conditions.
AI DISCLOSURE:
We may use artificial intelligence (AI) or other automated tools to support parts of our recruitment process. No automated tools make hiring decisions.
APPLY NOW:
If you are interested in finding out more, please contact us or submit your resume. Or, if you know someone who meets these qualifications, please forward this assignment.
ABOUT SYSTEMATIX:
Systematix is one of Canada’s largest privately-owned National Consulting and Resourcing firms. With offices across North America, we provide the highest-caliber consulting solutions to a diverse client base that includes all levels of government and private industry sectors.
Systematix is committed to creating a diverse, inclusive environment and is proud to be an equal opportunity employer.
Systematix. Results Focused. People Driven.
📌 Databricks Consultant (Canada)
🏢 Systematix
📍 Canada