21 Oct
|
Enterprise Solutions
|
Mississauga
21 Oct
Enterprise Solutions
Mississauga
Databricks Lead Developer
Location: Mississauga, Ontario, Canada
Employment Type: Permanent
Experience: 8+ Years
Job Summary
We are seeking an experienced Databricks Lead Developer to design, develop, and optimize scalable data engineering solutions using Azure Databricks. The ideal candidate will have solid hands-on expertise in PySpark, Python, SQL, and Azure data services, along with technical leadership experience.
Key Responsibilities
- Design and develop scalable ETL/ELT pipelines using Azure Databricks, PySpark, and Python.
- Implement Delta Lake, Delta Live Tables (DLT), and Unity Catalog.
- Build and optimize data pipelines using Azure Data Factory and ADLS Gen2.
- Implement CI/CD pipelines using Azure DevOps and Git.
- Optimize data processing performance, scalability, and cost.
- Ensure data quality,
governance, security, and operational excellence.
- Mentor developers and collaborate with cross-functional teams.
Mandatory Skills
- Azure Databricks, PySpark, Apache Spark, Python, and SQL
- Azure Data Factory (ADF) and ADLS Gen2
- Delta Lake, Delta Live Tables (DLT), and Unity Catalog
- Databricks Workflows
- CI/CD using Azure DevOps and Git
- Data Engineering, ETL/ELT, and Data Warehousing
Good-to-Have Skills Scala, Azure Synapse Analytics, Snowflake, Airflow, Databricks Asset Bundles, Auto Loader, Terraform, Medallion Architecture, Data Governance, and GenAI/RAG integrations.
📌 Databricks Lead Developer (Mississauga)
🏢 Enterprise Solutions
📍 Mississauga