Confluent is pioneering the foundational platform for real-time data streaming. Founded by the original creators of Apache Kafka, we enable forward-thinking organizations to connect applications, systems, and data streams, turning isolated data into actionable real-time intelligence. Our global engineering teams architect highly available, secure, distributed systems that operate at massive planetary scale.

We are seeking extraordinary technical leaders who are energized by solving complex distributed systems challenges and driven to create profound, measurable impact for thousands of enterprise customers worldwide. Confluent is pioneering the foundational platform for real-time data streaming. Founded by the original creators of Apache Kafka, we enable forward-thinking organizations to connect applications, systems, and data streams, turning isolated data into actionable real-time intelligence.

Our global engineering teams architect highly available, secure, distributed systems that operate at massive planetary scale. We are seeking extraordinary technical leaders who are energized by solving complex distributed systems challenges and driven to create profound, measurable impact for thousands of enterprise customers worldwide. Operating behind the scenes, it governs data and metadata compaction, intelligent log rewriting, retention policies, segment lifecycle management, asynchronous object garbage collection, and multi-tenant task orchestration. getting it right empowers seamless, zero-downtime, infinite-scale data streaming for the modern enterprise.

As a Senior Principal Engineer, you will define and drive the multi-year technical strategy for the Background Plane, shaping its pivotal role within the broader Next-Gen Kafka architecture. This is an individual contributor role with company-wide reach and executive-level scope.



You will lead high-ambiguity, highly visible initiatives spanning cloud storage, foreground serving engines, consensus layers, cloud infrastructure, control planes, and site reliability engineering—aligning senior technical leaders on mission-critical architecture calls while mentoring and sponsoring the next generation of principal talent.

Establish the long-term technical vision for the Background Plane, translating strategic business and product objectives into world-class distributed architectures, multi-year engineering roadmaps, and high-impact investment priorities.

Guarantee

End-to-End Correctness: Own the mathematical and operational correctness model for all data and metadata workflows (compaction, rewriting, retention, segment lifecycle, object GC), building bulletproof safety guarantees under extreme concurrency, partial network failures, and degraded dependencies. Spearhead architectural contracts across organizations—spanning data references, metadata interfaces, object life cycles, encryption models, and task schedulers—leading initiatives from initial framing to global production rollout.

Raise Operational Excellence: Define and enforce global engineering standards for fault tolerance, chaos testing, deep observability, canary safety, and incident mitigation, continually driving down systemic risk across reliability, cost, and velocity.

Serve as Senior Authority:



Act as the definitive technical authority during critical architectural reviews, high-severity incident responses, and complex technical trade-off discussions spanning product, infrastructure, and security.

Elevate Engineering Culture: Sponsor and mentor senior and staff engineers across the organization while representing Confluent's technical vision externally through keynotes, research publications, and open-source contributions. Master's Degree Required Technical And Skilled Expertise Proven track record of architecting, building, and operating mission-critical distributed systems at planetary scale, with deep expertise in storage engines, distributed log stores, stream processing, replication, or consensus protocols.

Strategic Technical Leadership: Demonstrated capability to articulate, execute, and deliver complex multi-year architectural roadmaps across multi-disciplinary engineering organizations.

Quality & SRE Rigor: Proven ability to establish measurable SLA/SLO metrics, durability targets, and automated recovery practices across global multi-tenant infrastructure fleets.

Influential Communication: Outstanding written and oral communication skills, with an ability to distill highly technical concepts for executive leadership and forge cross-team alignment through trust and expertise.

Preferred Technical And Professional Experience

Log-structured storage formats, custom storage engines, cloud object stores, or large-scale data re-writing pipelines. Fault-tolerant object lifecycle strategies designed to prevent data leakage, orphaned resources, or premature deletion. Chaos engineering, production traffic replay platforms, deterministic simulation testing, and large-scale load generation. Multi-cloud, multi-region deployment topologies across AWS, GCP, and Azure. #

📌 (Confluent) Senior Principal Engineer, Kafka (Ontario)
🏢 IBM
📍 Ontario