Build production-grade AWS data pipelines with Spark and PySpark as a Senior Data Engineer supporting Middle Office dataset integration and delivery.
This subcontractor role covers dataset onboarding, integration, and operationalization for analytics, reporting, and downstream applications. You will own the lifecycle from source analysis and solution design through implementation, deployment, and operational support. The work requires scalable data engineering across AWS and modern data lake technologies.
Key Responsibilities: Analyze source systems and onboard current datasets
Develop AWS pipelines using EMR, Glue, Athena, and S3
Automate workflows with Airflow scheduling and monitoring
Design schemas, partitioning, and data quality controls
Troubleshoot production data and performance issues
Requirements: Strong AWS data platform and pipeline experience
Advanced Apache Spark and PySpark expertise
Apache Hudi or Apache Iceberg experience
Strong Python, testing, Git, and software engineering practices
Terraform, CI/CD, and workflow orchestration experience
Apply Senior Data Engineering expertise across AWS, Spark, Airflow, and production data solutions for Middle Office functions.

#J-18808-Ljbffr

📌 Senior AWS Data Engineer (Ontario)
🏢 Luxoft
📍 Ontario