Role name: Senior Engineer Work Location: Canada (Remote) Job Description: Software Engineer (Spark / SparkSQL / Python / Scala) We are seeking a highly skilled Software Engineer with strong expertise in Apache Spark, SparkSQL, Python, Scala, Hive, and Parquet to build and maintain large-scale data processing solutions.

The ideal candidate will have experience developing distributed data applications, optimizing big data workloads, and building reliable data pipelines that support analytics, reporting, and machine learning initiatives.

Key Responsibilities: Design, develop, and maintain scalable data processing applications using Apache Spark and SparkSQL.

Build high-performance ETL and data transformation pipelines using Python and Scala.

Develop and optimize batch and near-real-time data processing workflows.

Work with large-scale datasets stored in Parquet, Hive, and distributed storage systems.

Write efficient SparkSQL queries and optimize execution plans for performance and scalability.

Design data models, partitioning strategies, and storage layouts to improve processing efficiency.

Collaborate with Data Engineers, Software Engineers, Data Scientists, and Product Teams to deliver data-driven solutions.

Troubleshoot performance bottlenecks and optimize resource utilization across distributed computing environments.

Implement automated testing, monitoring, and deployment processes for data applications.





Ensure data quality, reliability, security, and governance standards are met.

Participate in code reviews and promote engineering best practices.

Required Qualifications: Bachelor''s or Master''s degree in Computer Science, Engineering, or a related technical field. years of professional software engineering or data engineering experience.

Solid hands-on experience with: Apache Spark/SparkSQL/Python/Scala Experience building distributed data processing applications.

Strong understanding of Hadoop ecosystem technologies, including: Hive/Parquet/HDFS Experience developing and optimizing ETL pipelines processing large-scale datasets.

Strong SQL programming and query optimization skills.

Familiarity with Linux environments and shell scripting.

Experience using version control systems such as Git.

Strong problem-solving and debugging abilities.

Preferred Qualifications: Experience with cloud platforms such as AWS Experience with distributed computing and large-scale data architectures.

Familiarity with Airflow orchestration framework Experience working with data lakes and lakehouse architectures.

Knowledge of data formats and storage technologies: Iceberg / ORC / Avro Experience with containerization technologies such as Docker and Kubernetes.

Mandatory Skills: Apache Spark SparkSQL Python Scala Hive Parquet

📌 Senior Engineer (Mississauga)
🏢 eTeam
📍 Mississauga