This Senior Data Engineer will be responsible for designing, building, and optimizing high-performance data pipelines and platforms within a Databricks and AWS workplace. The team is currently in the middle of a major Databricks transformation initiative, focused on modernizing their data architecture while continuing to support and enhance components of an existing AWS-based data stack. This initiative is expected to run through the end of the year, with multiple key deliverables still in progress. As a result, this role will be heavily involved in both net-new development within Databricks and the ongoing support and evolution of AWS data infrastructure, including pipelines that support enterprise BI and reporting systems. The Senior Data Engineer will partner closely with Data Analytics leadership, including a recently established Head of Data & Analytics, as well as cross-functional stakeholders across business intelligence, reporting, and machine learning teams. This role will help bridge the gap between data engineering and advanced analytics by enabling reliable, production-ready data pipelines that support ML workflows and data-driven decision-making. This is a high-impact, senior-level position requiring strong ownership, technical depth, and the ability to operate independently. The ideal candidate will be comfortable stepping into an environment with active initiatives underway and contributing immediately to both delivery and longer-term data platform improvements.Key ResponsibilitiesDesign, build, and maintain scalable data pipelines using Python, Spark,



and DatabricksSupport and enhance an existing AWS-based data stack, ensuring performance, scalability, and reliabilityContribute to a large Databricks initiative, helping drive development and implementation effortsPartner with Data Analytics and BI teams to support enterprise reporting and analytics needsEnable and support machine learning workflows, including data preparation and pipeline integrationOptimize data processing performance and ensure strong data quality practicesCollaborate with stakeholders to translate business needs into technical data solutionsContribute to ongoing improvements in data architecture, tooling, and engineering best practicesREQUIRED SKILLS AND EXPERIENCE5+ years of experience in Data Engineering or a related fieldStrong hands‐on experience with Python and Apache SparkProven experience working in AWS environmentsHands‐on experience with Databricks and building production‐level data pipelinesExperience supporting BI/reporting platforms and analytics use casesExposure to or experience supporting machine learning workflows or ML pipelinesStrong understanding of data architecture, ETL/ELT processes, and distributed systemsAbility to operate independently in a senior‐level capacity and drive deliverablesNICE TO HAVE SKILLS AND EXPERIENCEExperience with AWS services such as Glue, Redshift, Lambda, or Step FunctionsFamiliarity with MLOps tools (e.G., SageMaker or similar platforms)Experience working in large‐scale enterprise data environmentsExposure to data governance, data quality, or observability frameworksExperience mentoring or guiding junior engineers #J-18808-Ljbffr

📌 Senior Data Engineer (Vaughan)
🏢 Insight Global
📍 Vaughan