Data Engineer – Data & Analytics Group

About KPI:

KPI Digital helps clients reach their business objectives by identifying required capabilities and by providing the proper data and analytics strategy and foundation allowing handling of all data challenges including enterprise-wide AI scaling. We specialize in Data & Analytics from Strategy, Architecture, Governance, Modern D&A; Platform, MDM and the entire D&A; Value Chain (Data Ingestion, Data Pipelines, Lakehouse, AI Modeling, DO/OR, BI, Deployment and Monitoring.

Performance Management and Digital Experience are two additional practices that complement the D&A; practice. In addition, to the Consulting Services, we offer Managed Services, Education and Training. Want to join a winning team?

The Role:

We are looking for a Data Engineer that combines Data Engineering, platform architecture and management expertise. The Data Engineer will:

Provide consulting services to our customers as the KPI Digital Lead Data Engineer
Define and maintain the KPI Digital Modern D&A; Platform Architecture. This D&A; Platform Architecture must:
Support real-time LakeHouse Architecture on Azure, AWS and GCP
Data feeds from business applications and sensors, IoT, PLC, Scada
Any type of workload from API on Insights, BI, AI ML/DL, NLP, DO/OR, Execution of ML models in real-time, batches, micro-batches or on edge
Data Ingestion in batch, micro-batches and streaming
Real-time dashboard
Respect highest security standards
Scalable, elastic and performant
Maintain AND Improve Accelerators to speed up the creation of Data Pipelines on major cloud providers
Participate and support the AI Pre-Sales activities
Build a Data Engineering team to meet the expected customers growth




Create Data Engineering content (articles, white papers and so on)
Speak in conferences, webinars to promote KPI Digital brand

The Ideal candidate is:

Goal oriented and possesses strong decision-making and problem-solving skills
Able to build a team of diverse talents to meet the customers expectations. This implies being quite involved in the recruiting and the active management of all Data Engineers (coaching, guiding, evaluating, setting objectives, reviews and so on)
Expected to work in close collaboration with the AI Team Lead, the Data Governance Lead and the VP of D&A;
A team lead, which means he/she not only assists his/her team, but the candidate is also responsible to bring his/her team to higher level of expertise
Able to devise training programs around the consulting mandates for his/her team to grow
Communicates and explains the plan, templates, approaches to his/her team and other teams so that he/she does not end-up being a bottleneck
Can communicate clearly in both languages (French and English) going technical when appropriate and is able to elevate the vocabulary for less technical folks such as clients, other team members, PM and so on
Excellent written and verbal communication skills for coordinating across teams (French and English)
Ability to discern fundamental and key aspects worth ‘fighting for’ versus the less ‘important’ ones.



The ideal candidate is ‘street smart’
Work with our customers’ stakeholders to identify opportunities for leveraging their existing D&A; platform toward a modern real-time LakeHouse
Coordinate with different functional teams to implement all D&A; assets with proper monitoring
Keep up to date with various up-to-date D&A; platform services and trends

Specific Skills:

Has implemented Lambda Architecture (may also have implemented KAPPA). And could create templates for a team of Data Engineers
Understands and did implement how to move data from Event Hub (or similar) to a raw zone, then an integrated zone and then to a consumption zone with replication to a Serving Layer with database like Synapse, Snowflake, BigQuery or Redshift (micro batches and streaming even in the cold layer)
Has been implementing the main 3 types of slowly changing dimensions in this technology or older ones
Has Data Engineering using DataVault
Has used Delta Lake
Knows how to stream data in the Data Lake using Databricks
Is very solid with Spark (to deal with Big Data situations) and can tune the environment using multiple nodes, is able to partition ‘DB’ and understand replicating the dimensions on multiple nodes
Can optimize the use of Azure resources to optimize costs and usage
Has some Azure administrator and cloud architecture experience
Can define the access rights management solution
Is an adept of CI-CD and can implement Azure DevOps
Can define real-time and batch orchestration
Has used these software (and implemented best practices for their usage): Chef, Jenkins, Terraform, Confluence, GIT, SVN, GitHub, Terraform, Bi

📌 Senior Data Engineer Ingénieur des données (Canada)
🏢 KPI Digital
📍 Canada