Description: Role: Data Engineer

Hybrid: 3 days a week in office

Max BR: ***

Job Summary An experienced Data Engineer with strong expertise in Big Data technologies to design, develop, and support enterprise-scale data platforms. The ideal candidate should possess hands-on experience in PySpark, Apache Spark, Kafka, Hadoop ecosystem components, and Apache NiFi, with a robust understanding of data ingestion, transformation, and real-time processing frameworks.

Key Responsibilities

Design, develop, and optimize scalable data pipelines using PySpark, Spark, Hadoop, and Apache NiFi.

Build and maintain batch and real-time data processing solutions.

Develop and support Kafka-based streaming applications and event-driven architectures.

Create and optimize ETL/ELT workflows for large-scale structured and unstructured datasets.

Develop complex SQL queries for data extraction, transformation, validation, and troubleshooting.

Implement data ingestion solutions from databases, APIs, files, and streaming sources.

Monitor, troubleshoot, and enhance the performance of Spark jobs and data pipelines.

Collaborate with architects,



business analysts, and development teams to deliver high-quality data solutions.

Support platform upgrades, deployments, testing, certification, and production releases.

Ensure data quality, governance, security, and operational excellence across data platforms.

Mandatory Skills

PySpark

Apache Spark (Spark SQL, DataFrames)

Apache Kafka

Hadoop Ecosystem (HDFS, Hive, YARN)

Apache NiFi

SQL

Python

Preferred Skills

Spark Streaming

Airflow / Oozie

Hive

Scala

Jenkins, Bitbucket, Git

JIRA, Confluence

Cloud Platforms (GCP/AWS/Azure)

Data Warehousing concepts and Dimensional Modeling

Custom Fields

Name: Business Vertical

Value: CL-BFSI Americas 2

Name: Requisition ID

Value: 10963265

Name: Parent

Value: IS-BFSI-Canada-1-Parent

Name: Work Location CDF

Value: ~Toronto 24th Floor~TCCI Canada Corporate~

Name: Child

Value: IS-BFSI-Canada-1-Group2

Name: Type of Asset

Value: None

Name: Work Assignment Location

Value: Hybrid

Name: Assets

Value: None

📌 Information Technology_CAN - CAN_Developer (Toronto)
🏢 Artech
📍 Toronto