As a Senior Databricks Data Platform Engineer, you will own the engineering, automation, and day-to-day health of our enterprise Databricks Lakehouse platform. This is a platform-first role: rather than building individual business pipelines, you will build the foundation that every data engineer, analyst, and data scientist depends on, including workspaces, compute, Unity Catalog governance, deployment pipelines, and the guardrails that keep the platform secure, reliable, and cost-efficient. That means codifying infrastructure and configuration, delivering changes through fully automated CI/CD, introducing agentic AI workflows that reduce manual operational effort, and continuously driving down compute and storage cost while improving performance. You will combine deep hands-on Databricks expertise with a FinOps and DevOps mindset, and you will measure your success in platform reliability, developer productivity, and dollars saved. You will partner with data engineering, data science, security, cloud infrastructure, and finance teams to set platform standards, guide adoption of new Databricks capabilities, and mentor engineers across the organization. Own the architecture, configuration, and lifecycle of client's Databricks environments across development, test, and production, including workspaces, account-level settings, networking, and identity integration Design and enforce compute strategy across all-purpose clusters, job clusters, serverless compute, SQL warehouses, instance pools, and cluster policies Manage platform upgrades, Databricks Runtime adoption, feature previews, and deprecations with minimal disruption to consuming teams Build reusable platform templates, starter projects, and self-service capabilities that let engineering teams onboard quickly and consistently Design, build,



and maintain CI/CD pipelines for Databricks code and configuration using Databricks Asset Bundles, Git folders, and Azure DevOps or GitHub Actions Codify platform infrastructure with Terraform (Databricks and cloud providers), ensuring every workspace, cluster policy, permission, and Unity Catalog object is version-controlled and reproducible Implement automated testing, code quality gates, security scanning, and promotion workflows from development through production Establish branching, release, and rollback standards for notebooks, jobs, Delta Live Tables / Lakeflow pipelines, and ML assets Automate routine administration such as user and group provisioning (SCIM), secret management, token rotation, and setting drift detection Agentic Workflows & AI-Enabled Operations Build AI agents and tools on Databricks (Mosaic AI Agent Framework, Model Serving, Vector Search) and integrate them with enterprise systems through Model Context Protocol (MCP) Use Copilot and Anthropic models to accelerate infrastructure-as-code, pipeline development, code review, and documentation Partner with data science and engineering teams to provide a governed platform foundation for GenAI, RAG, and ML workloads Own Databricks cost visibility and accountability using system tables (billing, usage, compute, and query history), tagging standards,



and chargeback or showback reporting Identify and deliver savings through cluster right-sizing, auto-termination, autoscaling tuning, spot and reserved capacity strategies, Photon and serverless evaluation, and workload consolidation Enforce cost guardrails through cluster policies, budget alerts, and compute governance that prevent waste without slowing teams down Storage & Performance Optimization Automate table maintenance using predictive optimization, OPTIMIZE, VACUUM, and retention policies to control storage growth and improve query performance Manage cloud object storage (ADLS Gen2 / S3) layout, lifecycle and tiering policies, and data retention aligned with business and compliance requirements Tune Spark and SQL workloads by analyzing query profiles, shuffle, skew, caching, and Photon usage to improve speed and reduce cost Identify and remediate small-file problems, orphaned data, unused tables, and redundant copies across the Lakehouse Implement secure-by-default platform controls, including private networking, IP access lists, encryption, secrets management, and least-privilege access Ensure auditable data access and lineage through Unity Catalog, audit logs, and system tables, with appropriate handling of PII and regulatory requirements Apply data classification, row
- and column-level security, and data masking standards across catalogs Define platform SLOs and build monitoring, alerting, and observability for jobs, clusters, warehouses, and pipelines Continuously evaluate new Databricks and cloud capabilities and lead proofs of concept that improve the platform About youYou are a hands-on platform engineer who thinks in terms of automation, reliability, and unit economics. You enjoy building the systems that make other engineers

📌 Databricks Data Platform Engineer (Canada)
🏢 UnivEdge Consulting
📍 Canada