SDET - Python / PySpark Data Test Engineer

100% Remote - Canada

Long Term Contract

:

Develop and enhance data testing frameworks using Python, PySpark, AWS S3, SQL, and automated testing tools. Focus on data validation, schema validation, ETL testing, and framework automation across multiple databases and data sources.

Core Skills:

- Programming / Data: Python, PySpark, SQL, Pandas
- Cloud: AWS S3
- ETL / Data: ETL, Informatica, DB2, Snowflake, Data Validation, Schema Validation, SCD Type 1
- Testing: Test Design, Data Testing, API Testing, Testing Techniques
- Automation: Pytest, ICEDQ, QuerySurge
- DevOps: GitHub, Jenkins
- Systems: Unix/Linux
- Framework: Test Automation Framework Development, Nested JSON, Multi-source/Target Validation

Key Responsibilities:





- Enhance Python/PySpark frameworks for data and ETL validation.
- Develop utilities to read and process data from AWS S3, including nested JSON.
- Implement schema, data, and SCD Type 1 validations.
- Transition framework execution from XLS-based drivers to Pytest.
- Validate data across multiple sources and targets including DB2 and Snowflake.
- Develop automated testing solutions using Pytest, ICEDQ, and QuerySurge.
- Perform API and data testing and support framework enhancements in Agile environments.
- Use GitHub, Jenkins, and Unix/Linux for development and CI/CD activities.

📌 SDET - Python / PySpark Data Test Engineer (Canada)
🏢 Centraprise
📍 Canada