- Designing and developing data pipelines from source to enduser

General Skills:

- Experience with Cloud data platforms, data management anddata exchange tools and technologies

- Experience with Commercial and opensource data and databasedevelopment and management and specializing in data storage setting andmanaging cloud Data As a Service (DaaS). application Database as a Service(DBaaS), Data Warehouses as a Service (DWaaS), and other storage platforms(both in the cloud and on-premise).

- Experience with Data pipeline and workflow development,orchestration, deployment, and automation and specializing in programming andpipelines to create and managing the dataflow and movement of data.

- Experience with Cloud data engineer must be familiar andexperienced with different programming languages and be able to integrate withmany different platforms to create data pipelines, automate tasks, and writescripts.

- Experience in DataOPS principle, best practice, andimplementation and Agile project development and deployment

- Experience in Continuous Integration/ContinuousDevelopment/Deployment (CI/CD) and Data provisioning Automation

- Experience with digital product, data analysis, dataexchange, data provisioning, and data security

- Extensive expert experience in designing/developing andimplementing data conversation and migration of VLD (Very large Data) of Onlineanalytical processing (OLAP) and online transaction processing (OLTP)environments to Cloud Software-as-a-service (SaaS), Platform-as-a-service(PaaS) and Infrastructure-as-a-service (IaaS) environments.

- Experience in design, development and implementation offact/dimension model, data mapping, data warehouse,



data lake and datalakehouse for enterprise

- Experience managing Cloud Data services for projectdelivery, including storage, repositories, Data Lake, Data Lakehouse, keyvault, virtual machine, disk, etc.

- Experience with structured, semi-structured, unstructureddata collection, ingestion, provisioning and exchange technological developmentof enterprise data warehouse and data lake and data lakehouse solutions andoperational support.

- Experience with DataOPS performance monitoring and tuning

- Excellent analytical, problem-solving and decision-makingskills; verbal and written communication skills; presentation skills;interpersonal and negotiation skills

- A team player with a track record for meeting deadlines

Requirements

Experience and Skill Set Requirements:

Must Haves:

- Design,build, automate and optimize complex data ETL/ELT processes usingDatabricks, Delta Live Tables (DLT) and equivalent.

- Developmentof unified data platforms using Databricks and Unity catalog or itsequivalent.

- Relationaldatabases (Oracle, MySQL, SQL Server), data modelling (relational &dimensional;), advanced SQL, query optimization, data replication,administration.

- BigData processing Frameworks (PySpark).





- Proficientin creating Azure Data Factory pipelines for copy activity and custom datapipelines in Databricks with CI CD (Azure DevOPS Git Integration) .

- Datamigration and data integration across different platforms Oracle to AzureData Lake, Oracle to Databricks, Oracle to Microsoft Fabric.

Skill Set Requirements:

Mandatory Skills and Experience (8+ years experience):

- Design,build, automate and optimize complex data ETL/ELT processes usingDatabricks, Delta Live Tables (DLT) and equivalent.

- Developmentof unified data platforms using Databricks and Unity catalog or itsequivalent.

- Relationaldatabases (Oracle, MySQL, SQL Server), data modelling (relational &dimensional;), advanced SQL, query optimization, data replication,administration.

- BigData processing Frameworks (PySpark).

- Proficientin creating Azure Data Factory pipelines for copy activity and custom datapipelines in Databricks with CI CD (Azure DevOPS Git Integration) .

- Datamigration and data integration across different platforms Oracle to AzureData Lake, Oracle to Databricks, Oracle to Microsoft Fabric.

Desirable Skills and Experience:

- Datalake design and development with focus on extracting, transforming datacoming from various data sources and loading into Lakehouse in medallionarchitecture.

- Advancedskills and hands-on experience with Oracle databases, Azure SQL Server andAzure Data Factory.

- PL/SQLfor data extraction, transformation, and loading, initial setup and otherETL experience management and support, such as troubleshooting,performance tuning, failover and recovery.

- Scriptinglanguages (Python, Unix shell, Scala).

#J-18808-Ljbffr

📌 11737 (Toronto)
🏢 Maarut
📍 Toronto