Key Responsibilities

- Design, develop, and maintain scalable data pipelines for batch and streaming data.

- Extract, transform, and load data from multiple sources using ETL/ELT processes.

- Develop and optimize SQL queries for data processing and analysis.

- Build data integration workflows using Python and relevant data engineering tools.

- Design and maintain data warehouses, data lakes, and data models.

- Ensure data quality, accuracy, consistency, and reliability.

- Monitor pipeline performance and troubleshoot data processing issues.

- Collaborate with Data Analysts, Data Scientists, and software development teams.

- Implement data validation, monitoring, logging, and error-handling mechanisms.

- Optimize data storage, processing performance, and infrastructure costs.

- Document data pipelines, workflows, and technical processes.

- Follow data security, access control, and governance practices.

Required Skills

- Robust knowledge of SQL, including joins, subqueries, CTEs, and window functions.

- Programming knowledge in Python.





- Understanding of ETL/ELT pipelines and data integration concepts.

- Knowledge of relational databases such as MySQL, PostgreSQL, or SQL Server.

- Understanding of data warehousing, data modeling, and database design.

- Familiarity with file formats such as CSV, JSON, and Parquet.

- Knowledge of data quality checks and pipeline troubleshooting.

- Understanding of distributed data processing concepts.

Preferred Skills

- Experience with Apache Spark or PySpark.

- Familiarity with Apache Airflow or other workflow orchestration tools.

- Knowledge of cloud platforms such as AWS, Microsoft Azure, or Google Cloud Platform.

- Experience with cloud data warehouses such as Snowflake, BigQuery, or Amazon Redshift.

- Familiarity with Apache Kafka for streaming data.

- Knowledge of Databricks and Delta Lake.

- Understanding of Docker, Git, and CI/CD pipelines.

- Familiarity with data governance, metadata management, and data security.

📌 Data Engineer (Canada)
🏢 Spait Infotech
📍 Canada