Role : Big Data Engineer (PySpark & Python)

Location: Mississauga, ON, Canada

Job Type: FTE

Salary : CAD 120K + Benefits

Job Summary

We are looking for an experienced Big Data Engineer with robust expertise in PySpark and Python to design, develop, and optimize scalable data processing solutions. The ideal candidate will have experience working with large datasets, building robust ETL/ELT pipelines, and supporting data integration and transformation initiatives in a cloud or big data environment.

Key Responsibilities

Design, develop, and maintain scalable data pipelines using PySpark and Python.

Build and optimize ETL/ELT workflows to process large volumes of structured and unstructured data.

Perform data cleansing, transformation, aggregation, and validation.

Develop efficient Spark jobs and troubleshoot performance and scalability issues.

Work with SQL, relational databases, and data warehouses to support analytics and reporting.

Implement data quality checks, monitoring, and error-handling mechanisms.

Collaborate with data engineers, architects, analysts, and business stakeholders to deliver data solutions.

Follow best practices for coding standards, documentation, testing, and data security.





Support cloud-based big data platforms and CI/CD processes where applicable.

Required Skills

Strong hands-on experience with Python and PySpark.

Solid understanding of Apache Spark, distributed computing, and big data processing.

Proficiency in SQL and experience with complex queries.

Experience developing ETL/ELT pipelines and data integration solutions.

Knowledge of data modeling, data warehousing, and data lake concepts.

Experience with performance tuning, debugging, and data quality management.

Familiarity with Git, unit testing, and Agile development methodologies.

Preferred Qualifications

Experience with cloud platforms such as AWS, Azure, or GCP.

Exposure to Databricks, Delta Lake, Apache Kafka, or Hadoop.

Experience with orchestration tools such as Airflow or Azure Data Factory.

Knowledge of CI/CD pipelines and automated deployment practices.

Education & Experience

Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field.

5+ years of experience in data engineering or big data development, with strong hands-on expertise in PySpark and Python.

📌 Big Data Developer (Mississauga)
🏢 TechNeptune Consulting
📍 Mississauga