17 Oct
|
Augmented
|
Toronto
About the role
Augmented is hiring a Data Engineer to build and run a modern lakehouse platform for one of our large enterprise clients in a regulated industry. You will design reliable batch and streaming pipelines, model data for analytics and AI use cases, and help retire legacy warehouse workloads.
You will join a small, senior delivery team that works directly with client stakeholders. The work spans greenfield builds and migrations, with a strong focus on data quality, governance and cost.
Key responsibilities
- Design and build lakehouse pipelines using a medallion (bronze/silver/gold) architecture on Delta Lake or Apache Iceberg
- Develop batch and streaming ingestion from databases, APIs, files and event streams (CDC, Kafka)
- Write production-grade PySpark and SQL transformations; tune Spark jobs for performance and cost
- Model curated data sets (dimensional, Data Vault or wide tables) for BI, analytics and ML teams
- Implement data quality checks, lineage and observability; own pipeline SLAs and incident response
- Apply governance and security controls: Unity Catalog or equivalent, row/column-level access, PII handling
- Migrate legacy warehouse and ETL workloads (e.g. Teradata, Oracle, SSIS, Informatica) to the lakehouse
- Build CI/CD for data code and infrastructure-as-code for platform resources
- Partner with client architects, analysts and data scientists; document designs and decisions
Required qualifications
- 4+ years in data engineering, with 2+ years on a lakehouse platform (Databricks, or Spark with Delta Lake/Iceberg)
- Strong Python (PySpark) and advanced SQL
- Hands-on with at least one major cloud: Azure, AWS or GCP (storage, compute, IAM, networking basics)
- Experience with orchestration such as Databricks Workflows, Airflow or Azure Data Factory
- Solid grasp of data modelling, partitioning, file formats (Parquet) and Spark performance tuning
- Git-based workflows and CI/CD (GitHub Actions, Azure DevOps or GitLab)
- Explicit communicator, comfortable working directly with enterprise client teams
- Legally entitled to work in Canada
Nice to have
- Databricks Certified Data Engineer (Associate or Professional), or equivalent cloud data certification
- dbt, Delta Live Tables / Lakeflow, or Spark Structured Streaming in production
- Terraform or Bicep for platform provisioning
- Experience in Life Sciences (GxP, clinical or commercial data) or Financial Services (risk, payments, regulatory reporting)
- Exposure to feature stores, vector search or data products for AI/ML
- Consulting or client-facing delivery background
Tech stack
- Platform: Databricks (Unity Catalog), Delta Lake / Apache Iceberg
- Languages: Python (PySpark), SQL
- Cloud: Azure (ADLS Gen2) or AWS (S3)
- Ingestion & streaming: Kafka / Event Hubs, CDC tools, Auto Loader
- Orchestration: Databricks Workflows, Airflow, Azure Data Factory
- Transformation: Spark, dbt, Delta Live Tables
- DevOps: Git, CI/CD pipelines, Terraform
- Consumption: Power BI, Tableau, Databricks SQL
Why Augmented
Augmented is an AI-native engineering delivery hub serving large enterprises in Life Sciences and Fintech, with teams in Toronto and the US.
- Work on high-impact data platforms for enterprise clients
- Small, senior teams with direct access to client decision-makers
- Exposure to AI-native delivery practices and modern tooling
- Compensation: [salary range to confirm] plus benefits
📌 Data Engineer Lakehouse (Toronto)
🏢 Augmented
📍 Toronto