Type:6-Month Contract (High likelihood of extension)Location:Toronto, ONWork Model:4 days/week onsite, Friday WFHPR: $80-100/hrRole OverviewWe are seeking aSenior Data Engineer (8-10+ years experience)to support a large-scale data platform transformation within the Total Fund Management (TFM) team.This role will focus onmigrating and modernizing existing Databricks-based pipelines to AWS (EMR Spark), with an initiallift-and-shift phase, followed byoptimization and redesign into scalable, consumable data products .This is ahighly autonomous, hands-on rolerequiring strong PySpark expertise, deep experience with distributed data systems, and the ability to navigate complex, multi-source datasets (including market and reference data vendors).Day-to-Day ResponsibilitiesMigrate existingDatabricks-based Spark pipelines to AWS EMR (Spark)Performlift-and-shift of ~50+ datasets, some with high complexity and multiple data sourcesRefactor and optimize data pipelines forperformance, scalability, and reliabilityStructure and store data usingParquet and IcebergformatsImprove and clean up legacy data pipelines built over several yearsDesign data with aconsumption-first mindset(e.G., partitioning strategies, access patterns, data usability)Collaborate with stakeholders to understand data requirements and translate into scalable solutionsEnsure production readiness includingmonitoring, orchestration, and deploymentWork independently to drive delivery from design through implementationKey ResponsibilitiesDevelop and optimizelarge-scale PySpark data pipelinesRebuild and enhance Spark workloads inAWS (EMR)Leverage tools such asAirflow, AWS Glue,



and Lake FormationImprove system performance and data quality across pipelinesEngage with business and technical stakeholders to align on data needsOwn delivery with minimal oversight in a fast-paced environmentMust-Haves8-10+ years of Data Engineering experience(senior-level profiles only)Strong hands-on expertise inPython and PySparkDeep experience withApache Spark in distributed environmentsProven experience working withlarge-scale, complex data pipelinesExperience withDatabricks(existing workplace)Strong knowledge ofParquet and Icebergdata formatsExperience withAWS data ecosystem (EMR preferred)Familiarity withAirflow, Glue, and Lake FormationStrong understanding ofparallel/distributed data processingAbility to work independently with strong problem-solving skillsExperience in ambiguous environments with evolving requirementsNice-to-HavesPrior experience incapital markets or investment managementExperience working withmarket data / reference data vendorsExperience designingdata products and consumption layersExposure to large-scaledata platform migrations or transformationsThe Company offers a total rewards package in accordance with all applicable federal, provincial, and local laws and requirements. Benefit eligibility and offerings vary based on role, employment status, and work location. For contractor positions, benefits are limited to those entitlements and protections required by applicable law, which may include (as applicable) vacation pay, public holidays, leaves of absence, and other legally mandated benefits or payments.#J-18808-Ljbffr

📌 Senior Data Engineer - $80 - $100 An Hour (Winnipeg)
🏢 Collabera
📍 Winnipeg