Build production-grade AWS data pipelines with Spark and PySpark as a Senior Data Engineer supporting Middle Office dataset integration and delivery.

This subcontractor role covers dataset onboarding, integration, and operationalization for analytics, reporting, and downstream applications. You will own the lifecycle from source analysis and solution design through implementation, deployment, and operational support. The work requires scalable data engineering across AWS and modern data lake technologies.

Key Responsibilities:

- Analyze source systems and onboard new datasets
- Develop AWS pipelines using EMR, Glue, Athena, and S3
- Automate workflows with Airflow scheduling and monitoring
- Design schemas, partitioning, and data quality controls
- Troubleshoot production data and performance issues

Requirements:

- Strong AWS data platform and pipeline experience
- Advanced Apache Spark and PySpark expertise
- Apache Hudi or Apache Iceberg experience
- Robust Python, testing, Git, and software engineering practices
- Terraform, CI/CD, and workflow orchestration experience

Apply Senior Data Engineering expertise across AWS, Spark, Airflow, and production data solutions for Middle Office functions.

📌 Senior AWS Data Engineer (Toronto)
🏢 Luxoft
📍 Toronto