Insight Global is hiring a Senior Data Engineer for a large retail client in Vancouver, BC. This role sits within the Data & Analytics organization and will support the Guest Data Lake platform, which enables trusted, scalable access to customer and guest data across digital, analytics, reporting, and personalization initiatives.
The ideal candidate brings deep experience building cloud-native data platforms, designing scalable data solutions, and supporting customer data domains using Azure, Databricks, PySpark, and SQL . You will work closely with Data Engineering, Analytics, Data Science, and business teams to deliver reliable, governed, and high-performing data products.
What You'll Do
Design and implement scalable data architectures, including batch, streaming, and lakehouse solutions
Enhance and support the Guest Data Lake platform to ensure reliable, high-quality customer data access
Build and optimize ETL/ELT pipelines for structured and semi-structured data sources
Enable analytics, personalization, reporting, and digital experience initiatives through customer data solutions
Lead migrations from legacy platforms to modern cloud-native architectures
Improve data quality, observability, monitoring, and governance frameworks
Partner with Data Science and Analytics teams to deliver ML-ready and analytics-ready datasets
Translate business requirements into scalable technical solutions
Drive CI/CD, Infrastructure-as-Code, and software engineering best practices
Participate in architecture reviews, technical design discussions, and platform roadmap planning
Mentor junior engineers through code reviews and technical guidance
Optimize platform performance, security, scalability, and cloud costs
Required Qualifications




5-8+ years of Data Engineering experience
Strong hands-on experience with Azure, Databricks, PySpark, and Unity Catalog
Advanced proficiency in SQL and Python
Experience supporting customer/guest data domains and customer-centric data platforms
Experience working with structured and semi-structured data, including NoSQL databases ( Cosmos DB preferred )
Solid understanding of Data Governance, Data Quality, and Data Observability best practices
Proven experience building and maintaining large-scale ETL/ELT pipelines and cloud-based data lakes
Experience with CI/CD , Terraform , and Infrastructure-as-Code practices
Experience contributing to architecture decisions and mentoring junior engineers
Preferred Qualifications
Retail, e-commerce, loyalty, or customer data platform experience
Experience with Spark Declarative Pipelines and Metric Views in Databricks
Experience supporting Machine Learning or AI-enabled data products
Understanding of Data Mesh architecture and domain-oriented data products
AWS experience
Experience with real-time streaming and event-driven architectures
Expand and evolve the Guest Data Lake to support new digital and customer experience initiatives
Design advanced data quality and observability frameworks
Support enterprise-wide data governance standards and implementation patterns
Implement Data Mesh principles and domain-driven ownership models
Build automation and self-service capabilities for data infrastructure provisioning
Support real-time and event-driven data processing solutions
Deliver scalable personalization, customer analytics, and guest engagement capabilities
Optimize cloud infrastructure for performance, scalability, reliability, and cost efficiency

#J-18808-Ljbffr

📌 Senior Data Engineer (British Columbia)
🏢 Insight Global
📍 British Columbia