29 Aug
|
KData AI
|
Brampton
Role Overview
We are seeking a Databricks Architect to lead the vision, design, and execution of large-scale, enterprise-grade Databricks Lakehouse architectures across multi-cloud and hybrid environments. In this role, you will advise executive leadership (CIOs/CTOs), drive pre-sales and technical solution design, and lead hands-on architecture for up-to-date data and GenAI platforms.
The ideal candidate brings 12+ years of technology experience, combining deep technical expertise in Databricks, multi-cloud strategy (AWS, Azure, GCP), TOGAF governance, data virtualization, and hands-on experience guiding enterprise modernization initiatives.
Key Responsibilities
1. Enterprise Solution Architecture & Design
- Lead architectural design and hands-on technical oversight for Databricks Lakehouse, Delta Lake, streaming pipelines, and AI-ready platforms.
- Define enterprise infrastructure reference architectures spanning cloud networking, compute, storage, security, observability, platform resiliency, disaster recovery, and compliance.
- Build end-to-end multi-cloud data integration layers and dynamic pipelines connecting legacy systems, cloud data lakes, and modern AI/ML frameworks.
- Establish governance models, architecture guiding principles, and standard frameworks (e.g., TOGAF) for enterprise platforms.
- Drive data virtualization and semantic layer strategies to streamline business intelligence and self-service analytics.
2. Client Engagement & Pre-Sales Leadership
- Serve as a trusted advisor to C-suite executives and CIOs on cloud modernization, data platform resiliency, cost optimization,
and AI-enabled operational transformations.
- Partner with Sales and Account teams on pre-sales activities, RFP responses, solution design, and high-value deal shaping.
- Conduct discovery workshops, present technical blueprints (SAD, DDD), and defend technical solutions to enterprise clients.
3. AI & Data Engineering Modernization
- Architect solutions incorporating Generative AI, automated document parsing/reformatting pipelines, advanced analytics, and machine learning models within the Databricks ecosystem.
- Oversee migrations from legacy databases/Big Data infrastructure (Hadoop, Oracle, Cloudera, Netezza) to modern cloud-native Lakehouses (AWS, Azure, GCP) using structured acceleration frameworks (e.g., AWS MAP).
- Establish robust security controls, encryption protocols, enterprise authentication mechanisms (Unity Catalog, Kerberos), and performance tuning for high-throughput environments.
Requirements Required Qualifications & Experience
- Experience: 12+ years of technology experience in Enterprise Infrastructure, Data Engineering, Multi-Cloud Architecture, and AI Strategy.
- Databricks Expertise: Strong hands-on architectural experience with Databricks Lakehouse, Delta Lake,
Spark, Unity Catalog, and AI/ML integrations.
- Multi-Cloud Expertise: Proven track record architecting and deploying solutions across AWS, Azure, and GCP platforms.
- Legacy Modernization: Solid background transitioning traditional database/Hadoop environments (Oracle RAC/ASM, Cloudera/Hortonworks, Netezza, HBase) into cloud-native architectures.
- Domain Exposure: Architecture experience supporting industries such as Healthcare, Insurance & Risk, Banking & Finance, Media, Retail, or Life Sciences.
Technical Stack & Skills
- Databases & Data Platforms: Databricks, Delta Lake, Snowflake, Redshift, BigQuery, ADLS, S3, Blob Storage, Oracle (RAC/Grid), Aurora, Netezza, Postgres, MySQL, SQL Server, HBase.
- Big Data & Data Integration: Apache Spark, Hadoop Ecosystem (HDFS, Hive, YARN, Sqoop, Oozie, Pig, Atlas, Ranger), Kafka, Azure Data Factory, AWS Glue, NiFi, Denodo (Data Virtualization), Informatica (IICS/PowerCenter), Syncsort.
- AI & Advanced Analytics: Generative AI pipeline design, Dataiku, AtScale, PowerBI.
- Observability, Operations & Tools: Grafana, SumoLogic, Git/GitStash, Jira, Bamboo, DB Visualizer.
- Languages & Scripting: Python, SQL, PL/SQL, Unix Shell Scripting, Perl Scripting.
- DevOps & Infrastructure: Terraform, AWS CloudFormation (CFT), CodeDeploy, CI/CD pipelines, Linux (RedHat), Windows AD.
Education
- Master’s or Bachelor’s degree in Business Administration, Computer Science, Software Engineering, or a related technical discipline.
📌 Databricks Architect (Cloud, Data & AI) (Brampton)
🏢 KData AI
📍 Brampton