27 Oct
|
Celebal Technologies
|
Canada
27 Oct
Celebal Technologies
Canada
Job Title: Databricks Architect
Location: Canada
Work Mode: Remote
Job Type: Contract / Full-Time
Certification: Databricks Certification preferred
Job Summary
We are seeking an experienced Databricks Architect to design and implement scalable, secure, and high-performance data lakehouse solutions. The ideal candidate should have strong expertise in Databricks, Medallion Architecture, Azure/AWS cloud platforms, SQL, PySpark, Python, and Databricks Auto Loader.
Key Responsibilities
- Design and implement enterprise-scale data architectures using Databricks.
- Develop scalable data pipelines using SQL, PySpark, and Python.
- Implement Medallion Architecture, including Bronze, Silver, and Gold layers.
- Build incremental and streaming data ingestion pipelines using Databricks Auto Loader.
- Integrate Databricks with Azure or AWS cloud services.
- Optimize Spark workloads, SQL queries, cluster configurations, and data processing performance.
- Implement Delta Lake solutions for reliable data storage and processing.
- Establish data governance, security, and access control best practices.
- Provide technical leadership and architectural guidance to data engineering teams.
- Support production deployments, troubleshooting, and performance optimization.
Required Skills and Qualifications
- Strong experience with Databricks architecture and Lakehouse solutions.
- Hands-on expertise in Medallion Architecture.
- Proficiency in SQL, PySpark, and Python.
- Practical experience with Databricks Auto Loader.
- Experience with Azure Databricks or AWS Databricks.
- Solid understanding of Apache Spark, Delta Lake, and ETL/ELT pipelines.
- Experience in data pipeline optimization and production-grade data engineering.
- Knowledge of data governance, security, and scalable architecture design.
- Strong communication, analytical, and problem-solving skills.
Preferred Qualifications
- Databricks certification preferred.
- Experience with Unity Catalog and data governance.
- Knowledge of Delta Live Tables (DLT) or Lakeflow Declarative Pipelines.
- Experience with CI/CD, Git, and infrastructure-as-code tools.
- Experience with cloud migration and enterprise data platform modernization.
- Previous experience leading technical teams or defining data architecture standards.
Ideal Candidate The ideal candidate will be a hands-on technical architect who can design end-to-end Databricks solutions, guide engineering teams, and deliver scalable data platforms using modern cloud and lakehouse technologies.
📌 Databricks Architect (Canada)
🏢 Celebal Technologies
📍 Canada