We are looking for a highly motivated 6-8 years of ETL Test Automation Engineer with solid expertise in Azure Data Factory (ADF), Azure Databricks, Python, SQL, and ETL Testing.
Key Responsibilities:
- Support ETL/ELT pipelines using Azure Data Factory and Azure Databricks.
- Optimize data ingestion frameworks and Data Lake processing pipelines.
- Implement data quality checks, data reconciliation, and validation frameworks.
- Monitor and troubleshoot data pipeline failures, performance bottlenecks, and production issues.
- Perform end-to-end ETL testing across source and target systems.
- Execute source-to-target validation, reconciliation, schema validation, and data integrity checks.
- Create and execute SQL-based validation scripts for large datasets.
- Validate transformation logic, business rules, and incremental data loads.
- Identify, track, and support closure of ETL and data quality defects.
- Support SIT, UAT, Regression, and Production validation activities.
- Automate regression and reconciliation activities to reduce manual effort.
- Integrate automated validation scripts into CI/CD pipelines.
- Drive continuous improvement through automation and AI-assisted testing practices.