Job Title: Bigdata Engineer

Duration: 6+ months contract to hire

Location: Mississauga Ontario

Looking for a Senior Big Data Engineer to design, build, and optimize large-scale data pipelines and distributed data systems that power critical business intelligence across the organization. Operating in a hybrid model, you will work within a high-performing engineering team where your expertise in PySpark, the Hadoop ecosystem, and streaming data platforms will directly shape the reliability and performance of Citi's data infrastructure.

6-8 years of professional developer experience is required,

Responsibilities:

Build and maintain scalable data pipelines using PySpark within a Big Data environment to process and transform large volumes of structured and unstructured data.

- Knowledge / experience with DataBricks.
- Design and develop solutions across the Hadoop ecosystem — including Hive, HDFS, Sqoop, Spark, Impala, and Scala — to enable efficient data ingestion, processing, and storage.
- Develop and manage real-time and batch data workflows using streaming data platforms, ensuring high availability and low-latency data delivery.
- Write complex SQL queries to extract, validate, and analyse data across distributed systems, supporting data-driven decision-making.
- Design and implement data models and data architecture patterns aligned with data warehouse principles, ensuring scalability, accuracy, and consistency.




- Automate pipeline scheduling and orchestration using shell scripting and Autosys, reducing manual intervention and improving operational reliability.
- Independently identify, assess, and resolve technical risks and data issues in a timely manner, maintaining system integrity across the data platform.
- Experience in BI Platforms (preferable Tableau) is a nice to have.

Required Qualifications & Skills

- Hands-on expertise in PySpark and Big Data processing, with the ability to build and optimise distributed data workflows at scale.
- Practical knowledge of the Hadoop ecosystem, including Hive, HDFS, Sqoop, Spark, Impala, and Scala, applied in a production environment.
- Proficiency in complex SQL query development for data analysis, transformation, and validation across large datasets.
- Solid understanding of distributed systems architecture and how data flows across interconnected processing layers.
- Demonstrated knowledge of data modelling and data design, with familiarity in data warehouse concepts and dimensional modelling techniques.
- Competence in shell scripting and job scheduling using Autosys or equivalent workflow automation tools.
- Strong analytical and problem-solving ability, with a track record of working independently to diagnose and resolve complex data engineering challenges.
- Transparent and effective communication skills, with the ability to articulate technical concepts to both technical and non-technical audiences.

📌 Big Data Consultant (Mississauga)
🏢 Kelly
📍 Mississauga