IDR is seeking a Data Engineer (Databricks) to join one of our top clients in a remote, direct-hire opportunity. This role is focused on the continued development and optimization of an enterprise lakehouse built on Azure Databricks. The ideal candidate has deep, hands-on Databricks expertise, experience across multiple areas of the platform, and a robust understanding of leveraging Databricks Genie and AI-driven analytics to support business initiatives.
-

Location:
Remote – USA (MT, CT, or ET) or Ontario, Canada
- Terms: Direct-hire (FTE)
- Sponsorship: Not supported Position Overview for the Data Engineer (Databricks):
- Develop, maintain, and optimize an enterprise lakehouse architecture using Azure Databricks.
- Own multiple aspects of the Databricks environment, including data ingestion, pipelines, transformations, security, governance, and data quality.
- Leverage Databricks Genie (conversational analytics) to improve data accessibility and support AI-driven business initiatives.
- Prepare, structure, and optimize enterprise data for AI and natural-language analytics applications.
- Identify opportunities to leverage AI for process automation, workflow optimization, and innovative data solutions beyond code generation.
- Collaborate with a small team of data engineers and architects to develop scalable solutions and establish Databricks best practices.



Requirements for the Data Engineer (Databricks):
- 4+ years of hands-on Azure Databricks experience, with deep knowledge of its tools, features, and capabilities.
- Demonstrated experience across multiple areas of Databricks beyond pipeline development, including ingestion, security, user access, data quality, and governance.
- Experience developing, maintaining, and optimizing enterprise lakehouse environments.
- Hands-on experience with Databricks Genie for conversational analytics , including enabling natural-language queries and AI-driven data exploration.
- Experience preparing and transforming data to support AI initiatives, with an understanding of how data should be structured for AI consumption.
- Proven ability to independently own Databricks solutions and explain specific technical implementations, processes, and tools used. Nice to Have:
- 5+ years of overall data engineering or data warehousing experience.
- Strong Python, T-SQL, and ANSI SQL skills.
- Experience with Agile methodologies, DevOps concepts, and Git.
- Experience leveraging AI for business process automation and workflow improvements.
- Background working on smaller data engineering teams with broad technical responsibilities. Compensation Details: $145,000-165,000 CAD

📌 Databricks Engineer (Ontario)
🏢 IDR
📍 Ontario