Design batch and streaming pipelines at Index Exchange using Spark, Kafka, and Flink for billions of daily events. Build reliable data systems at scale.

As a Staff Data Engineer, you will evolve the data processing and pipeline layer supporting reporting, billing, and real-time data streams. You will transform high-volume event data into clean, queryable, production-grade datasets. The work spans distributed processing, SQL, data modeling, workflow orchestration, and performance optimization.

Key Responsibilities: Design and operate Spark, Kafka, and Flink pipelines

Build canonical datasets and reporting tables

Maintain Airflow DAGs for scheduling and backfills

Optimize runtime, throughput, compute efficiency, and scalability

Define data processing standards across teams

Requirements: Robust experience in data engineering at scale

Expertise in distributed processing frameworks, SQL, and data modeling

Experience with Airflow, Kafka, or streaming systems

Experience managing Kubernetes

Ability to lead projects and debug complex data issues

Bring staff-level data engineering leadership to Index Exchange through scalable pipelines, streaming systems, and optimized data platforms.

📌 Staff Data Engineer Pipelines (Toronto)
🏢 Index Exchange
📍 Toronto