13 Nov
|
Capgemini
|
Mississauga
13 Nov
Capgemini
Mississauga
Job Title: Spark Developer (Java)
Experience:
4 to 8 Years
Location:
Any Employment Type: Full-time Job Summary We are seeking a highly skilled Spark Developer with strong Java expertise to design, develop, and optimize large-scale data processing applications. The ideal candidate will have hands-on experience with Apache Spark, Core Java, distributed computing, and big data technologies. The candidate will work closely with data engineers, architects, and business stakeholders to build scalable and high-performance data solutions.
Key Responsibilities
• Design, develop, and maintain scalable data processing applications using Apache Spark and Java.
• Develop batch and near real-time data pipelines for large datasets.
• Optimize Spark jobs for performance, memory management, and resource utilization.
• Write efficient and reusable code using Java 8/11 features.
• Work with distributed data processing frameworks and big data ecosystems.
• Collaborate with cross-functional teams to gather requirements and deliver data solutions.
• Troubleshoot production issues and provide performance tuning recommendations.
• Implement best practices for coding, testing, deployment, and monitoring.
• Participate in code reviews and technical design discussions.
• Ensure data quality, reliability,
and scalability of applications. Required Skills
• Strong programming experience in Core Java (Java 8 or above).
• Hands-on experience with Apache Spark (Spark Core, Spark SQL, DataFrames, Dataset APIs).
• Good understanding of distributed computing concepts.
• Experience with HDFS, Hive, Hadoop Ecosystem.
• Solid SQL skills and experience with relational databases.
• Experience with data ingestion and transformation frameworks.
• Familiarity with Linux/Unix environments.
• Knowledge of REST APIs and microservices architecture.
• Experience with Git and CI/CD processes. Preferred Skills
• Experience with Spark Streaming, Kafka, Scala, or PySpark.
• Exposure to cloud platforms such as AWS, Azure, or GCP.
• Experience with Airflow, Oozie, or similar workflow orchestration tools.
• Understanding of Delta Lake, Iceberg, or Data Lake architectures.
• Knowledge of containerization tools such as Docker and Kubernetes. Qualifications
• Bachelor's or Master's degree in Computer Science, Information Technology, or related field.
• 4+ years of software development experience.
• 2+ years of hands-on experience developing applications using Apache Spark and Java.
📌 Spark Developer (Mississauga)
🏢 Capgemini
📍 Mississauga