06 Sep
|
Saxon Global
|
Calgary
06 Sep
Saxon Global
Calgary
Role: Senior Databricks Data Engineer
Location: Calgary, Canada
- Real-Time & Batch Integration: Configure and manage continuous data ingestion pipelines using Apache Kafka event streams for real-time market data, alongside scheduled batch processing of Parquet files and Delta Lake tables.
- Medallion Data Modeling: Architect and implement Databricks Lakehouse data models (Bronze/Silver/Gold layers) tailored for energy commodities, ensuring high performance, data lineage, scalability, and compute efficiency.
- Cross-Desk Reporting Delivery: Collaborate with Front, Middle, and Back Office stakeholders to transform raw transactional data into curated datasets supporting critical reporting for risk (VaR), PnL, scheduling, and regulatory compliance.
- Databricks Platform Architecture:
Serve as a Subject Matter Expert in designing, configuring, and securing Databricks environments, including Unity Catalog governance, cluster optimization, CI/CD deployment workflows, and platform performance tuning.
- Data Engineering & Optimization: Build scalable ETL/ELT workflows using PySpark and SQL, optimizing large-scale distributed data processing across cloud-native lakehouse architectures.
- Governance & Reliability: Implement monitoring, alerting, and data quality frameworks to ensure reliable, auditable, and compliant enterprise data pipelines.
📌 Senior Databricks Data Engineer (Calgary)
🏢 Saxon Global
📍 Calgary