06 Nov
|
ML arteka
|
Winnipeg
06 Nov
ML arteka
Winnipeg
Who we areML arteka is one of the fastest-growing IT Services and Consulting companies in Canada and serves some of the most iconic brands, including Google, Loblaws, Staples, Telus, Bell, Rogers, CIBC, BMO, CMLS, Fossil, Fitbit and more. This reflects our reliability in delivering what we promise and the quality we bring to every engagement. Join our growth-driven culture and help us lead, innovate, and simplify. The Role :Build the source-to-Silver ingestion layer for a complex data and AI platform transformation at a regulated financial institution. Own the design and delivery of 30+ source connectors across low, medium, and high complexity tiers — file-based, SQL-based, and API-based — on a medallion architecture running on AWS. You are not a pipeline configuration resource. You are an engineer who understands what the data means, where it breaks, and why, and you build connectors that hold up in production under schema drift, late arrivals, and source quality gaps. Design, build, test, and own source connectors from S3 landing to validated Silver records. S3 trigger configuration, Bronze layer write (immutable raw copy with provenance metadata), Silver layer transformation (schema enforcement, field mapping, type casting, null and range checks, deduplication, bad record quarantine), and source arrival monitoring feeding the pipeline audit table. Translate source-to-target mapping specifications — provided by the client data analyst team — into connector logic that handles both single-field mappings and multi-field derivations: conditional expressions, arithmetic combinations across multiple source fields, and lookup-based enrichment. Investigate and document source data behavior during the discovery phase: field semantics, undocumented transformations currently embedded in legacy code,
frequency and format variability, and data quality gaps that must either be replicated or explicitly flagged in the recent platform. Build and maintain the source arrival monitoring layer: expected arrival windows per source, SLA breach alerting, missed file notification, and source health dashboard feeding CloudWatch. Work directly with the client's embedded data engineers and data analyst in a co-development model — peer-reviewing their connector code against platform standards, contributing to theirs, and maintaining shared code quality across a jointly owned codebase. Participate in the parallel run: investigate reconciliation breaks traced to the Bronze or Silver layer, distinguish source defects from connector defects, and produce a root cause statement the QA lead can include in the Phase Gate Report. Write and maintain connector unit tests and integration tests as first-class deliverables, not afterthoughts — your connectors are tested before they reach QA, not handed over for testing. Strong Python engineering skills — you write production-quality code, not scripts. You know the difference between code that works once and code that holds up under bad data, late files, and schema changes three months into a project. Experienced building data ingestion pipelines on AWS: S3-triggered processing, Glue ETL jobs or equivalent, schema enforcement, and pipeline observability.
You read a source data dictionary and immediately know which fields will cause problems, which data types will drift, and which business rules embedded in the legacy system are about to become your transformation problem. Experience with orchestration frameworks (MWAA, Step Functions, Airflow, or Control-M): you understand how your connector fits into the dependency graph and what happens downstream when it fails. Detail-oriented on data provenance — you build Bronze layers that make forensic investigation possible, not just fast processing. Operates effectively in regulated environments where every data movement needs a traceable audit record and every schema change needs a documented rationale. What to expectBesides Competitive Salary and working for one of Canadas Best Managed and Top 50 Technology Companies, you should expect: Positive Environment: recognition coupled with a flexible environment that celebrates diversity, inclusivity and encourages everyone to achieve their best Growth & Development: opportunities to grow professionally and personally through mentorship, training, and development programs Exclusive Perks: discounts on shopping, the gym, insurance, mobile phone plans, not to mention an office smoothie bar and enough coffee to make you vibrate You will get health, dental, vision, disability, travel insurance, and much more all covered by us Unforgettable Events: team bonding, BBQs, office lunches (pizza, always pizza), gala dinners, and any other excuse to have fun Inclusion and equal opportunity employmentML arteka is an equal opportunity employer committed to diversity and inclusion. We are pleased to consider all qualified applicants for employment without regard to race, color, religion, sex
📌 Sr Data Engineer (Winnipeg)
🏢 ML arteka
📍 Winnipeg