Required Skills & Experience
5+ years of experience in

Python development and data engineering.
Robust hands-on experience with

PySpark and Apache Spark

.
Experience developing

ETL/ELT data pipelines using PySpark.
Solid knowledge of

Python programming

, including data structures, functions, OOP, and error handling.
Experience working with

SQL and relational databases.
Good understanding of

Data Warehousing concepts and data modeling.
Experience with large-scale data processing and performance optimization.
Experience with

Spark SQL, DataFrames, RDDs, and Spark transformations/actions

.
Hands-on experience with cloud platforms such as

AWS, Azure, or GCP

.
Experience working with data storage technologies such as

S3, ADLS, Azure Data Lake, or HDFS

.




Experience with

Git and CI/CD practices.
Robust understanding of data quality, validation, and troubleshooting.

Preferred Skills
Experience with

Databricks and Delta Lake.
Experience with

AWS Glue, EMR, Azure Data Factory, or similar data engineering tools

.
Knowledge of

Kafka or other streaming technologies

.
Experience with

Airflow or other workflow orchestration tools.
Knowledge of Docker/Kubernetes is a plus.
Experience with Snowflake or other cloud data warehouses.
Exposure to Agile/Scrum development methodologies.

📌 Pyspark Python Developer Ontario (Canada)
🏢 Avance Consulting
📍 Canada