12 Aug
|
Foilcon
|
Toronto
Join to apply for the Software Developer ETL 9222-0715 role at FoilconNote: This hybrid contract role is three (3) days in office. Candidates' resumes must include first and last name.OverviewThe Office of the Public Guardian and Trustee (OPGT) requires a Senior Software Developer - ETL to perform data migration activities from the existing OPGT legacy applications to the current Dynamics 365 OPGT solution and data warehouse/lakehouse.ResponsibilitiesAs a member of the data migration team, you will be responsible for migrating the data from the legacy applications to the Dynamics 365 OPGT solution and the new data warehouse/lakehouse. A high-level list of deliverables for the data migration team follows:Data Analysis: analyze the existing data in the legacy applications, understand its structure, quality, and relationships and help in designing an appropriate migration strategy; Data Mapping and Transformation: map the data elements from the legacy application to the corresponding entities and fields in Dynamics 365 CE, F&O and Data Lakehouse. Handle necessary data transformations, ensuring compatibility and consistency between the legacy data and the target system; Data Extraction: help extract the required data from the legacy application, develop and implement extraction processes to retrieve data from various sources, such as databases, files, APIs, or other relevant legacy systems; Data Cleansing and Validation: cleanse and validate the extracted data to ensure its accuracy, completeness, and consistency. Help with identifying and resolving data quality issues, performing deduplication, and applying business rules to ensure the integrity of the migrated data; Data Migration Strategy and Execution: review the present migration strategy that outlines the overall approach, sequence, and timeline for migrating the data from the legacy application to Dynamics 365 using a delta-load approach; execute the migration plan efficiently, managing data transfers and ensuring minimal disruption to ongoing operations; Data Testing and Quality Assurance: conduct thorough testing to verify the accuracy and integrity of the migrated data; define test cases, perform data reconciliation, and address any issues or discrepancies that arise during the testing phase; develop KPIs to report on the progress, completeness and quality of the data migration effort; Documentation: document the entire data migration process, including data mapping rules, transformation logic, migration scripts, and any specific configurations; Ongoing Support: provide post-migration support, analyze and address data-related issues or questions; help optimize data management processes in the new environment; Data Lakehouse: help expand the current data lakehouse implementation and help build a gold layer for reporting; conduct data analysis and profiling; develop data transformation and cleansing pipelines; implement data integration and harmonization; Other duties as assigned.The Consultant Will Also Be Required ToComplete work and achieve milestones within the assigned deadlines; Notify the Cluster/Ministry project Manager in writing of any issues or other material concerns related to the Assignment Deliverables,
as soon as he/she becomes aware of them; Submit Deliverables for the Cluster/Ministry approval as they are completed.Co mply with the Ontario Government and the Cluster/Ministry security procedures and practices; Co mply with the Ontario Government and the Cluster/Ministry architecture/technology standards and best practices; Co mply with the Ontario Government and the Cluster/Ministry Conflict of Interest and Confidentiality Guidelines; Provide knowledgeand skill transfer to a designated Cluster/Ministry staff; andCo mplywith the Ontario Government I&IT Directive, Operational Policy on the I&IT Project Gateway Process, and other applicable Guidelines, Standards and Procedures.Experience and Skillset RequirementsMandatory Requirements5+ years of proven working experience in an ETL role; strong understanding ofETL principles, including data extraction, transformation, and loading processes; knowledge of common ETLdesign patterns. Understanding of data pipeline architectures, Azure workflow orchestration tools, and concepts related to data ingestion, transformation, and movement.Proficiency in Azure Data Factory, Azure Synapse workspaces, PolyBase, including knowledge of pipeline creation, data flows, integration runtimes, triggers, and monitoring.Knowledge of integration technologies commonly used with Dynamics, such as DataVerse / Common Data Service (CDS), Data Entities, and APIs.Nice To HaveAzure cloud certifications (e.G. Azure fundamentals, Azure Data Engineer associate, Azure Database Administrator associate)Experience with PowerApps platform, Power Automate, Dynamics CE and F&ODesired Skills And Experience5+ years of proven working experience in an ETL role; strong understanding ofETL principles, including data extraction, transformation, and loading processes; knowledge of common ETLdesign patterns. Understanding of data pipeline architectures, Azure workflow orchestration tools, and concepts related to data ingestion, transformation, and movement.Experience in integrating various data sources and systems, both on-premises and in the cloud, using Azure ETL services or other ETL toolsKnowledge of integration technologies commonly used with Dynamics, such as DataVerse / Common Data Service (CDS), Data Entities, and APIs.Expertise in data transformation techniques, such as data cleansing, aggregation, enrichment, and normalization using Azure cloud technologiesUnderstanding of data quality management practices, including data profiling, data validation, and error handling within ETL processes.Understanding of data governance principles, data privacy regulations and experience working with high-sensitivity data, and knowledge of best practices for data security and compliance in Azure.Ability to monitor and troubleshoot ETL processes, optimize query performance, and implement efficient data processing techniques in Azure.Proficiency in Azure Data Factory, Azure Synapse workspaces, PolyBase, including knowledge of pipeline creation, data flows, integration runtimes, triggers,
and monitoring.Strong SQL skills and experience working with Azure SQL Databases and Dataverse; good understanding of Azure storage concepts and technologies.Proficiency in scripting languages like Python, and experience with Azure-specific scripting using PowerShell or Azure CLI.Expert in data manipulation languages (T-SQL, PL/SQL), data definition languages, physical database design, data modelling, query performance analysis & tuningFamiliarity with version control systems (e.G., Azure Repos) and collaboration tools (e.G., Azure DevOps) for managing code, tracking changes, and collaborating with team members.Experience with continuous integration/continuous deployment (CI/CD) processes around DevOps, data workflows, Synapse workspaces.Experience with SQL Server Management Studio, Azure data management tools, XRM toolbox, data modeling tools (preferably, ERWIN).Resumes EvaluationCriteria 1: Data Migration, ETL - 40 PointsDemonstrated experience with ETL development, data pipelines, workflow orchestration and data ingestion, transformation, and movementDemonstrated experience in integrating various data sources and systems, both on-premises and in the cloud, using Azure ETL services or other ETL toolsDemonstrated experience working with Azure Data Factory, Azure Synapse workspaces, PolyBase, including knowledge of pipeline creation, data flows, integration runtimes, triggers, and monitoring.Demonstrated experience with data manipulation languages (T-SQL, PL/SQL), data definition languages, query performance analysis & tuningDemonstrated experience with SQL Server, Oracle, Azure SQL DatabasesDemonstrated experience with data modeling tools (preferably, ERWIN)Demonstrated experience in scripting languages like Python and with Azure-specific scripting using PowerShell or Azure CLI.Experience with software development lifecycleExperience with data modeling, physical database design, data flow diagramsCriteria 2: Azure Platform - 20 PointsExperience with Azure Data Factory (ADF) and Synapse WorkspacesDemonstrated experience Azure data management tools, DevOps, Synapse StudioExperience in Azure resource configuration and administration such as Azure Data Lake, Blob Storage, Key Vault, Application Insight resources, resource groups and subscriptions.Familiar with Azure cloud platformAzure cloud certificationsCriteria 3: Dynamics 365 - 20 PointsDemonstrated experience working with integration technologies commonly used with Dynamics, such as DataVerse / Common Data Service (CDS), Data Entities, and APIs.Demonstrated experience with PowerApps platform, Power Automate, Dynamics CE & F&OCriteria 4: DevOps and CI/CD - 20 PointsDemonstrated experience with continuous integration/continuous deployment (CI/CD) tools and processes around DevOps, data workflows, Synapse workspaces.Knowledge TransferWhat Needs To Be KTDocument the tasks executed and in process of execution as a member of the data migration team using the approach and tools required by the project manager.To WhomProject Manager / Team membersWhen1:1 meetings / team meetings / documentation in DevOps Wiki and boards throughout the duration of the project life cycle.Job DetailsSeniority level: Mid-Senior levelEmployment type: ContractJob function: Engineering and Information TechnologyIndustries: IT Services and IT ConsultingLocation: Toronto, Ontario, CanadaSalary: CA$140,000 - CA$180,000 per year#J-18808-Ljbffr
📌 Software Developer Etl 9222-0715 - C$140,000 - C$180,000 A Year (Toronto)
🏢 Foilcon
📍 Toronto