22 Oct
|
Hexaware
|
Canada
Real-time data pipelines using Azure Databricks, PySpark, Kafka, Azure Data Factory (ADF), and Azure Data Lake Storage (ADLS), enabling reliable ingestion and transformation of 10M+ records daily.
- Developed and optimized SQL and PySpark transformations within medallion-style ETL workflows, improving processing efficiency, maintainability, and data quality for downstream analytics.
- Identified requirement gaps and data quality issues stemming from upstream data sources and solutioned root-cause fixes by implementing validation rules, refining transformation logic, and collaborating with stakeholders to improve downstream data reliability.
📌 Data Integration Lead (Canada)
🏢 Hexaware
📍 Canada