Big Data: Apache Spark, Hadoop Scala/Java

What are the top 3 skills required for this role?

- Agentic AI, LLM, RAG Pipelines

- MLOps Automation (CI/CD, data validation, model deployment, monitoring, drift detection)

- GitHub/GitHub Actions, Docker, Kubernetes

Responsibilities

Build and maintain scalable batch and streaming data pipelines for ingestion, transformation, and curation

Design and optimize data models, feature stores, and storage patterns for AI/ML workloads

Implement DataOps and MLOps automation (CI/CD, data validation, model deployment, monitoring, drift detection)

Design, build, and optimize agentic AI systems and LLM-powered applications (RAG pipelines, agent orchestration)

Develop and integrate AI services into secure, production-grade environments

Design and implement AI systems architecture best practices and standards across the organization

Build scalable, resilient cloud and on-premise systems for hosting AI/LLM applications

Provide infrastructure design, optimization, and monitoring support

Translate business requirements into technical solutions through cross-functional collaboration

Ensure code quality, system reliability, scalability, and observability





Expectations / Skillset Required

Advanced Python proficiency with strong software engineering fundamentals (version control, testing, code reviews)

Proficiency in SQL and distributed computing (Spark, distributed systems)

Deep experience with data modeling and feature engineering across the ML lifecycle

Hands-on experience building LLM applications, RAG pipelines, and agentic AI frameworks (LangChain/LangGraph preferred)

Expertise designing distributed systems at enterprise scale

Robust CI/CD and DevOps knowledge (GitHub/GitHub Actions, Docker, Kubernetes)

Experience with event streaming platforms (Kafka)

Demonstrated ability to design and deploy hybrid cloud and on-premise systems (AWS, Azure)

Excellent collaboration and communication skills across engineering, research, and product teams

Self-directed problem-solving mindset and ability to navigate enterprise complexity

Master's degree in Computer Science, Software Engineering, or related field with 5+ years in data/software/ML/AI engineering (preferred)

UI development expertise with React and JavaScript (a plus)

📌 Gen AI Developer (Halifax Regional Municipality)
🏢 Ju0026amp;M Group
📍 Halifax Regional Municipality