21 Oct
|
Miratech
|
Toronto
Miratech helps visionaries change the world. We are a global IT services and consulting company that brings together enterprise and start-up innovation. Today, we support digital transformation for some of the world's largest enterprises. By partnering with both large and small players, we stay at the leading edge of technology, remain nimble even as a global leader, and create technology that helps our clients further enhance their business. We are a values-drivenorganizationand our culture of Relentless Performance has enabled over 99% ofMiratech'sengagements to succeed by meeting or exceeding our scope, schedule, and/or budgetobjectivessince ourinceptionin 1989.
Miratech has coverage across 5 continents andoperatesin over 25 countries around the world. Miratech retainsnearly 1000full-time professionals, and our annual growth rate exceeds 25%.
Job Description
We are looking for a Senior Data Architect to design the end-to-end data platform for APEX. The platform ingests contact centre telephony and quality data from the Genesys A3S Snowflake database and Verint Workforce Management (WFM) feeds, and delivers it through a Bronze/Silver/Gold (Medallion) architecture on Databricks into a Kimball-style reporting layer.
You will own the architecture: ingestion strategy and cadence per source, incremental loading and batch control, data integrity standards, and the Gold-layer dimensional model. You will also partner with the existing CCARS pipeline team through parallel run and decommissioning of the legacy pipelines.
This role suits a hands-on Senior or Lead Data Architect who can turn complex reporting requirements into robust, auditable data solutions and explain architectural trade-offs to both engineers and business stakeholders.
Working Model: This is a hybrid role requiring 2 days per week in a downtown Toronto office.
Responsibilities:
- Design the Bronze/Silver/Gold data flow for the new Genesys A3S Snowflake and Verint sources, with transparent standards for ingestion, transformation, validation, and consumption.
- Define ingestion cadence and watermarking strategy per source, based on each source's true data freshness.
- Design the Gold-layer Kimball star schema (facts, dimensions, grain, SCD Type I/II) to support sales, telephony, agent activity, and quality reporting.
- Choose between pure lakehouse and hybrid lakehouse-to-data-warehouse patterns based on source characteristics, volumes and performance needs.
- Build and optimize Databricks and Delta Lake pipelines, and set cost and performance practices for cluster sizing, job versus all-purpose compute, and Photon.
- Design ingestion from the external Genesys Snowflake source, including secure connectivity, incremental extraction and source-to-target reconciliation.
- Integrate Verint and other vendor feeds delivered as flat files, SFTP drops or REST APIs, normalizing inconsistent formats and tolerating schema drift.
- Design API ingestion that handles pagination, rate limits, asynchronous submit-poll-retrieve jobs and per-entity fan-out at scale.
- Define data integrity checks, batch control,
idempotency and failure-recovery patterns for each pipeline, including row-count and checksum reconciliation and late-arriving data handling.
- Design orchestration with Azure Data Factory, Databricks Workflows or equivalent, with independent batch tracking and failure isolation per source feed.
- Write and review production-grade transformation logic in T-SQL, Spark SQL and Python/PySpark.
- Document architecture decisions and trade-offs for engineering and business audiences.
- Partner with the existing CCARS pipeline team on parallel-run validation, cutover planning and legacy pipeline decommissioning.
Qualifications
- 10+ years of designing data warehouse and lakehouse solutions.
- 5+ years of hands-on experience with Databricks/Delta Lake and Snowflake or API-driven ingestion.
- Proven experience designing Medallion (Bronze/Silver/Gold) architectures, including the trade-offs between pure lakehouse and hybrid lakehouse-to-DW designs.
- Strong Delta Lake expertise: Delta tables, ACID transactions, Z-ordering, OPTIMIZE/VACUUM, and Databricks workload tuning at scale.
- Expert-level T-SQL and Spark SQL, including complex joins, window functions, query tuning, indexing and partitioning.
- Deep knowledge of relational databases (SQL Server preferred) and of OLTP versus OLAP design.
- Track record enforcing data integrity across pipeline boundaries: referential integrity, deduplication, reconciliation, checksum/hash validation, and detecting upstream schema drift.
- Experience consuming REST APIs at scale, including pagination, rate limiting, async job patterns and endpoint-specific freshness.
- Experience ingesting fro
📌 Senior Data Architect (Toronto)
🏢 Miratech
📍 Toronto