21 Oct
|
Infosys
|
Mississauga
21 Oct
Infosys
Mississauga
We are seeking an experienced Data Engineer to design, develop, and support enterprise data engineering and analytics solutions across up-to-date cloud platforms. The ideal candidate will have hands‑on experience with the complete project life cycle — from requirements gathering and design through development, testing, deployment, and post‑production support — along with strong technical expertise in Python, Snowflake, Azure Data Factory (ADF) or Microsoft Fabric, and advanced SQL. This role requires strong UNIX/Linux shell scripting skills to build, optimize, and troubleshoot complex data pipelines supporting business‑critical data and analytics initiatives.
Required Qualifications
- Candidates authorized to work for any employer in Canada without employer‑based visa sponsorship are welcome to apply. Infosys is unable to provide immigration sponsorship for this role at this time.
- Candidate must be located within commuting distance of Mississauga, ON / Calgary, AB / Burnaby, BC or be willing to relocate to the areas.
- Bachelor's degree or foreign equivalent required from an accredited institution. Will also consider three years of progressive experience in the specialty in lieu of every year of education.
- At least 4 years of Information Technology experience, including hands‑on data engineering or ETL/ELT development.
Preferred Qualifications
- Strong hands‑on experience with Python for data pipeline development, automation, and scripting (e.g., pandas, PySpark, or similar libraries).
- Experience building and managing data pipelines and warehousing solutions in Snowflake (e.g., Streams, Tasks, Pipes, warehouses, role‑based access, performance/cost optimization).
- Hands‑on experience with Azure Data Factory (ADF) and/or Microsoft Fabric for cloud‑based data integration, orchestration, and transformation.
- Strong proficiency in SQL (complex queries, joins, stored procedures, performance tuning) on any major RDBMS or cloud data warehouse (Snowflake, Azure SQL, Synapse, Oracle, SQL Server, etc.).
- UNIX/Linux shell scripting for automation, job scheduling, and file/data handling.
- Experience with ETL/ELT performance tuning and troubleshooting.
- Knowledge of job scheduling/orchestration tools (e.g., Airflow, Control-M, Autosys, ADF triggers/pipelines) is a plus.
- Familiarity with data warehousing concepts, dimensional modeling, and data governance practices.
- End‑to‑end exposure to project phases: initiation, requirement analysis, design, build, testing, UAT, deployment, and warranty/support.
- Experience working in Agile/Scrum or Waterfall project environments.
- Ability to prepare and review technical design documents, test plans, and deployment plans.
- Experience coordinating with cross‑functional teams (business, QA, infrastructure, DBA) to ensure on‑time, quality delivery.
- Familiarity with change management, version control (e.g., Git), and release management practices.
Key Responsibilities
- Participate in the complete project life cycle (PLC) — requirement gathering, analysis, design,
development, unit testing, UAT support, deployment, and post‑go‑live support.
- Design, develop, and maintain data pipelines and workflows using Python, Snowflake, and Azure Data Factory (ADF) or Microsoft Fabric.
- Build and manage cloud‑native data integration and transformation pipelines, including ingestion, staging, and curated/analytics‑ready layers in Snowflake.
- Analyze business and functional requirements and translate them into technical design documents (source‑to‑target mappings, pipeline design specs).
- Develop and optimize complex SQL queries, stored procedures, and views for data extraction, transformation, and validation.
- Write and maintain UNIX/Linux shell scripts for job scheduling, file handling, automation, and workflow orchestration.
- Perform data profiling, data quality checks, and error handling within data pipelines.
- Conduct performance tuning of pipelines, warehouses, and SQL queries to improve throughput and reduce processing time/cost.
- Collaborate with business analysts, DBAs, QA teams, and project managers throughout all SDLC/PLC phases.
- Prepare and maintain technical documentation, including design documents, unit test cases, and deployment/run‑books.
- Support migration/modernization activities onto Snowflake, ADF, and/or Microsoft Fabric platforms.
- Troubleshoot production issues, perform root cause analysis, and implement permanent fixes.
- Adhere to coding standards, version control practices, and change management/release processes.
- Provide effort estimates, status updates, and risk assessments to project managers and stakeholders.
Nice To Have
📌 Technology Lead (Mississauga)
🏢 Infosys
📍 Mississauga