Big Data Engineer

Mississauga, ON Hybrid(3 days office per week)

Fulltime

JD

- Develop and maintain scalable data pipelines using PySpark and Spark SQL for processing large datasets efficiently.

- Write clean, reusable, and optimized code in Python for data manipulation, analysis, and automation tasks.

- Design and implement ETL workflows to extract, transform, and load data from various structured and unstructured sources.

- Collaborate with data engineers, analysts, and stakeholders to understand data requirements and deliver solutions.

- Optimize Spark jobs for performance tuning, resource utilization, and minimizing execution time.

- Work with distributed computing frameworks to process and analyze big data in a cloud or on-premises environment.





- Develop and maintain unit tests to ensure the accuracy and reliability of data pipelines and transformations.

- Utilize Spark SQL for querying and managing large datasets stored in distributed systems like Hadoop or cloud storage.

- Monitor and troubleshoot data pipeline issues, ensuring reliability and timely delivery of data.

- Stay updated with the latest advancements in PySpark, Spark SQL, and big data technologies to improve existing systems.

Pay: $100,000.00-$130,000.00 per year

Benefits

- Versatile schedule

Experience

- Big Data: 6 years (required)

- SQL: 5 years (required)

- PySpark: 1 year (required)

Work Location: Hybrid remote in Mississauga, ON (L5B)

📌 Big Data Engineer (Mississauga)
🏢 Citrine Solution
📍 Mississauga