Required Skills & Experience

- 5+ years of experience in

Python development and data engineering.

- Strong hands-on experience with

PySpark and Apache Spark

.

- Experience developing

ETL/ELT data pipelines using PySpark.

- Strong knowledge of

Python programming

, including data structures, functions, OOP, and error handling.

- Experience working with

SQL and relational databases.

- Good understanding of

Data Warehousing concepts and data modeling.

- Experience with large-scale data processing and performance optimization.

- Experience with

Spark SQL, DataFrames, RDDs, and Spark transformations/actions

.

- Hands-on experience with cloud platforms such as

AWS, Azure, or GCP

.

- Experience working with data storage technologies such as

S3, ADLS, Azure Data Lake, or HDFS

.





- Experience with

Git and CI/CD practices.

- Robust understanding of data quality, validation, and troubleshooting.

Preferred Skills

- Experience with

Databricks and Delta Lake.

- Experience with

AWS Glue, EMR, Azure Data Factory, or similar data engineering tools

.

- Knowledge of

Kafka or other streaming technologies

.

- Experience with

Airflow or other workflow orchestration tools.

- Knowledge of Docker/Kubernetes is a plus.

- Experience with Snowflake or other cloud data warehouses.

- Exposure to Agile/Scrum development methodologies.

📌 PySpark Python Developer (Ontario)
🏢 Avance Consulting
📍 Ontario