08 Nov
|
Randstad Deutschland
|
Toronto
08 Nov
Randstad Deutschland
Toronto
We are seeking an experienced Senior Software Developer - ETL to lead the technical design, development, and sustainment of scalable, production-grade data pipelines, reusable data services, and integration frameworks. In this role, you will enable enterprise artificial intelligence, analytics, and business intelligence initiatives by transforming raw structured and unstructured data into governed, high-quality, and AI-ready assets. Leveraging advanced Python engineering, complex SQL, cloud-based data platforms, and the Azure Data and AI ecosystem, you will build automated ETL/ELT pipelines for AI/ML and Generative AI workloads—including data ingestion, feature engineering, model training and inference datasets, and Retrieval-Augmented Generation (RAG) grounding data—while operationalizing enterprise integration patterns to ensure robust system reliability, performance, and maintainability.
Location: Toronto, ON (Hybrid – 3 days a week onsite) Duration: 6-month contract.
Advantages
- Modern AI Data Stack: Engineer data pipelines for Generative AI, RAG, and Azure AI Foundry platforms.
- Production Engineering Scope: Build modular, scalable ETL/ELT pipelines using disciplined software engineering standards.
- High-Impact Collaboration: Partner directly with data architects, AI engineers, and cross-functional technology teams.
- Balanced Hybrid Schedule: Work comfortably with a 3-day onsite model based in Toronto.
Responsibilities
- Design, build, test, and maintain robust, scalable ETL/ELT data pipelines and integration workflows across diverse enterprise data sources.
- Develop production-grade data pipelines supporting AI/ML and Generative AI workloads, including ingestion, transformation, feature engineering,
model training/inference datasets, and RAG grounding data.
- Build reusable data services, pipelines, and integration components to accelerate onboarding of new enterprise data sources and eliminate redundant processes.
- Ingest, clean, transform, and prepare structured and unstructured data for downstream consumption by data scientists, machine learning models, and analytics platforms.
- Implement data architecture, data modeling, and integration standards established by enterprise architecture teams.
- Integrate datasets across cloud environments, relational databases, enterprise applications, APIs, and file-based data stores.
- Embed comprehensive data quality checks, validation rules, error handling, and automated reconciliation into all pipeline stages.
- Partner with Data Architects, AI/ML Engineers, Data Scientists, and product teams to translate business and analytical requirements into production-ready data pipelines.
- Troubleshoot, debug, and optimize complex pipeline performance bottlenecks, data latency issues, and pipeline runtime failures.
- Maintain clear technical documentation, architectural specifications, runbooks, and source code using software engineering best practices.
Qualifications
- AI/ML & GenAI Data Engineering: Significant hands‑on experience designing and developing scalable,
production-grade data pipelines for AI/ML and Generative AI workloads, including data ingestion, transformation, feature engineering, training/inference datasets, RAG/grounding data, and AI/ML pipeline integration (Strictly Required).
- Advanced Python, SQL & ETL/ELT: Advanced hands‑on expertise in Python, SQL, ETL/ELT architecture, APIs, data integration, transformation, and cloud data platforms, applying disciplined software engineering practices to build reliable and maintainable solutions (Strictly Required).
- Azure Data & AI Ecosystem: Solid working knowledge of the Azure data and AI ecosystem, Azure Machine Learning, and Microsoft Foundry / Azure AI Foundry, including familiarity with ML pipelines, feature stores, model testing, inference workflows, and monitoring (Strictly Required).
- Data Integration & Storage: Demonstrated experience integrating complex data flows across relational databases, APIs, cloud‑based data repositories, and enterprise applications (Strictly Required).
- Data Governance & Architecture: Solid understanding of data modeling, data quality validation, metadata management, lineage, data security, and enterprise lifecycle governance (Strictly Required).
- Education & Professional Experience: Degree in Computer Science, Information Systems, Engineering, Data/Analytics, or a related discipline, or equivalent practical experience (Strictly Required).
Nice-to-Have Assets
- Prior data engineering or ETL development experience within the public sector.
- Relevant cloud or data engineering certifications (e.g., Azure Data Engineer Associate, Azure Solutions Architect).
- Experience with distributed computing frameworks (such as Databric
📌 Senior Software Developer (Toronto)
🏢 Randstad Deutschland
📍 Toronto