Our client needs a Senior ETL PySpark developer to work on projects related to data platform and pipelines.

Services

This Senior Developer will work on creating, operating (run processes) and maintaining the portfolio of products assigned to them as well as, test, prepare deployment packages, deploy code/script changes, support user tests and inquires and perform or follow up on application administration tasks required for the products maintenance.

Deliverables

- Understand systems and Business processes related to the products in their assigned portfolio
- Works closely with our client analytical teams to develop, maintain and support products using Python and/or R and PySpark programming languages.
- Provides support and guidance on creating top-quality clean code and participates in code review.
- Analyses Software deficiencies/problems and support to fix them
- Prepares documentation, reports, and presentations related to projects and activities.
- Shares knowledge and expertise across the organization.
- Process operational tasks in these products like running scripts in test and prod environments, change config files, etc, to support scheduled releases of the products
- Perform maintenance tasks in these products like changes to roll-over to next Fiscal Year and others
- Develop code changes as per approved product updates requirements to these products




- Test thoroughly the changes and maintain test scenarios as applicable
- Perform deployment to test environments and support QA team (as applicable) and users testing
- Support users inquiries
- Communicate well with Business users as well as other groups to ensure all tasks required for the products maintenance are covered
- Perform or follow up with application administration tasks as required
- Prepare deployment packages and required release tickets to deploy the changes
- Follow up on deployment tasks with all groups involved

Required Skills:
1. Python and PySpark programing languages to manipulate data and draw insights from large datasets
2. Experience with AWS EMR, AWS Cloud
3. Hands-on programming skills, able to write modular, maintainable code, preferably Python & SQL. Good understanding of SQL, dimensional modeling, and analytical big data warehouses
4. Experience with Apache Airflow
5. Experience with Python, PySpark programming in the context of data processing / data science applications
6. Profound knowledge and experience with AWS Redshift and Oracle database
7. Profound knowledge and experience in Oracle PL/SQL
8. Profound knowledge and experience in shell scripting in Linux/Unix environment. Experience in creating and maintaining documentation of project ETL design, technical artifacts and ETL setting

This is a contract role and work type is REMOTE.

📌 Sr. Software Developer, ETL, PySpark (Canada)
🏢 IT Connex
📍 Canada