26 Oct
|
Celebal Technologies
|
Canada
26 Oct
Celebal Technologies
Canada
Job Title: Databricks Architect
Location:
Canada Work Mode: Remote
Job Type
Contract / Full-Time Certification: Databricks Certification preferred Job Summary We are seeking an experienced Databricks Architect to design and implement scalable, secure, and high-performance data lakehouse solutions. The ideal candidate should have robust expertise in Databricks, Medallion Architecture, Azure/AWS cloud platforms, SQL, PySpark, Python, and Databricks Auto Loader.
Key Responsibilities
• Design and implement enterprise-scale data architectures using Databricks.
• Develop scalable data pipelines using SQL, PySpark, and Python.
• Implement Medallion Architecture, including Bronze, Silver, and Gold layers.
• Build incremental and streaming data ingestion pipelines using Databricks Auto Loader.
• Integrate Databricks with Azure or AWS cloud services.
• Optimize Spark workloads, SQL queries, cluster configurations, and data processing performance.
• Implement Delta Lake solutions for reliable data storage and processing.
• Establish data governance, security, and access control best practices.
• Provide technical leadership and architectural guidance to data engineering teams.
• Support production deployments, troubleshooting, and performance optimization. Required Skills and Qualifications
• Strong experience with Databricks architecture and Lakehouse solutions.
• Hands-on expertise in Medallion Architecture.
• Proficiency in SQL, PySpark, and Python.
• Practical experience with Databricks Auto Loader.
• Experience with Azure Databricks or AWS Databricks.
• Strong understanding of Apache Spark, Delta Lake, and ETL/ELT pipelines.
• Experience in data pipeline optimization and production-grade data engineering.
• Knowledge of data governance, security, and scalable architecture design.
• Strong communication, analytical, and problem-solving skills. Preferred Qualifications
• Databricks certification preferred.
• Experience with Unity Catalog and data governance.
• Knowledge of Delta Live Tables (DLT) or Lakeflow Declarative Pipelines.
• Experience with CI/CD, Git, and infrastructure-as-code tools.
• Experience with cloud migration and enterprise data platform modernization.
• Previous experience leading technical teams or defining data architecture standards. Ideal Candidate The ideal candidate will be a hands-on technical architect who can design end-to-end Databricks solutions, guide engineering teams, and deliver scalable data platforms using up-to-date cloud and lakehouse technologies.
📌 Databricks Architect (Canada)
🏢 Celebal Technologies
📍 Canada