Design and implement Lakehouse architectures using Delta Lake.
Build and optimize batch and streaming pipelines using Apache Spark (PySpark/Scala).
Implement data quality, monitoring, and CI/CD best practices.
Optimize cluster performance and cost efficiency within Databricks.
Provide technical leadership, conduct code reviews, and mentor junior engineers.
Required Experience
8+ years in data engineering, with robust hands-on experience in Databricks.
Advanced SQL and data modeling expertise.
Deep experience with Spark performance tuning and large-scale data processing.
Solid understanding of cloud platforms (AWS, Azure, or GCP).
Proven ability to design and deliver production-grade data platforms.
J-18808-Ljbffr