04 Aug
|
CGI
|
Halifax Regional Municipality
04 Aug
CGI
Halifax Regional Municipality
Position Description: We are seeking an experienced Data Engineer to design, build, integrate, and support modern cloud-based data platforms and scalable data engineering solutions.
The successful candidate will work across Google Cloud Platform (GCP), Microsoft Azure, and Databricks environments to develop robust data pipelines, data integration frameworks, and cloud-native solutions that support enterprise analytics, reporting, and operational workloads.
This role requires strong expertise in cloud data services, distributed data processing, event-driven architectures, data integration, and Dev
Ops practices.
The ideal candidate is passionate about delivering scalable, secure, and high-performance data solutions in an Agile environment.
Your future duties and responsibilities: Design, develop, and maintain cloud-based data platforms and data engineering solutions.
Build scalable ETL/ELT pipelines using modern cloud technologies and data processing frameworks.
Develop and optimize data solutions using GCP, Azure, and Databricks services.
Integrate enterprise applications, databases, APIs, and event-streaming platforms.
Develop data models and optimize large-scale data storage and processing solutions.
Build and maintain RESTful APIs, microservices, and event-driven architectures.
Implement secure, reliable, and high-performing cloud-native data solutions.
Collaborate with cross-functional teams including architects, developers, analysts, and business stakeholders.
Support CI/CD pipelines, automated deployments, and Dev
Ops best practices.
Perform unit, integration, and automated testing to ensure solution quality.
Produce technical documentation and contribute to Agile ceremonies and project delivery Required qualifications to be successful in this role: 5+ years of experience in Data Engineering or a related role.
Strong experience building cloud-based data platforms and ETL/ELT pipelines.
Hands-on experience with Google Cloud Platform (GCP) services, including Big
Query, Dataflow, Data
Proc, Cloud SQL, Cloud Spanner, Cloud Storage, and Composer.
Experience with Databricks, including Lakehouse architecture, Unity Catalog, and MLflow.
Experience with Azure Data Factory and Azure SQL.
Robust SQL skills and experience with relational databases such as PostgreSQL, Oracle, and Google Spanner.
Experience with event streaming technologies including Kafka and Google Cloud Pub/Sub.
Proficiency in Python, Apache Spark, Hadoop, and Talend.
Experience developing RESTful APIs, microservices, and event-driven architectures using Java and Spring Boot.
Experience with cloud-native technologies including Docker, Kubernetes (AKS/GKE), Cloud Run, and Azure Functions.
Experience with Git, Git
Hub, Bitbucket, CI/CD pipelines, JIRA, and Confluence.
Knowledge of Agile delivery, automated testing, secure coding, and software development best practices.
Preferred Qualifications Experience building enterprise-scale cloud-native data platforms.
Knowledge of data governance and metadata management.
Experience with CI/CD automation and Infrastructure as Code.
Experience working within Agile/Scrum teams.
Excellent analytical, problem-solving, and communication skills.
Technical Environment Cloud: Google Cloud Platform (GCP), Microsoft Azure Data: Databricks, Big
Query, Cloud SQL, Cloud Spanner, Azure SQL, Azure Data Factory Programming: Python, Java, Apache Spark, Hadoop Integration: Kafka, Cloud Pub/Sub, REST APIs, Spring Boot Containers: Docker, Kubernetes (AKS/GKE), Cloud Run Dev
Ops: Git
Hub, Bitbucket, Git
Hub Actions, Bitbucket Pipelines Methodology: Agile, CI/CD, Clean Architecture, Domain-Driven Design (DDD). CGI is providing a reasonable estimate of the pay range for this role.
The determination of this range includes factors such as skill set level, geographic market, experience and training, and licenses and certifications.
Compensation decisions depend on the facts and circumstances of each case. A reasonable estimate of the current range is $60,$,.
This role is an existing vacancy. #LI-OA1 Use of the term engineering in this job posting refers to the technical sense related to Information Technology (IT) and does not imply that the individual practices engineering or possesses the requisite license as prescribed by the applicable provincial or territorial engineering regulator.
We are seeking individuals with expertise in IT engineering-related functions, but licensure from an engineering regulator is not a prerequisite for this position.
Engineering is a regulated profession in Canada which is restricted in terms of use of titles and designation.
Skills: Java Microsoft Azure Data Factory Python Spring Boot SQL
📌 Data Engineering Specialist (Halifax Regional Municipality)
🏢 CGI
📍 Halifax Regional Municipality