18 Nov
|
Virtusa
|
Toronto
Job Title: AWS Data Engineer
Location: PCS CA~TORONTO
Years of Experience: 5-7 Years
Job Summary: We are seeking an experienced AWS Data Engineer with a strong background in data engineering and data warehousing. The ideal candidate will have hands-on expertise in core AWS services and a proven track record of designing and optimizing scalable data pipelines. This role requires collaboration with cross-functional teams to deliver high-quality data solutions.
Responsibilities
- Design, build, and optimize scalable batch and real-time data pipelines using PySpark and Python.
- Develop and manage data warehousing solutions using Amazon Redshift and S3 data lake architecture.
- Implement workflow orchestration using Apache Airflow or AWS Step Functions to manage dependencies and automate tasks.
- Integrate CI/CD processes for deployment automation and infrastructure support.
- Ensure data quality and security by implementing validation, encryption, IAM roles, and governance policies.
- Optimize SQL queries for performance on large-scale enterprise data.
- Collaborate with architects and stakeholders to deliver effective data solutions.
Mandatory Skills
- 7+ years of experience in Data Engineering and Data Warehousing.
- Hands-on expertise with AWS services: AWS Glue, Amazon Redshift, Amazon S3, EMR, and Athena.
- Proficient in designing and optimizing data pipelines using PySpark and Python.
- Strong experience in Amazon Redshift modeling and S3 data lake architecture.
- Experience with workflow orchestration tools such as Apache Airflow or AWS Step Functions.
- Advanced SQL optimization skills for large-scale data.
Preferred Skills
- Experience with CI/CD integration and deployment automation.
- Knowledge of data governance and security best practices.
Qualifications
- Bachelor's degree in Computer Science, Information Technology, or a related field.
- Solid analytical and problem-solving skills.
- Excellent communication and collaboration abilities.
📌 Lead Software Engineer (Toronto)
🏢 Virtusa
📍 Toronto