05 Nov
|
Tekfortune
|
Calgary
05 Nov
Tekfortune
Calgary
calgary, AB based, working from office 5 days a week
Role Summary:
We are looking for an experienced Kafka Platform Administrator / Kafka Developer with strong expertise in Apache Kafka, Kafka on-premises and cloud environments, and Red Hat OpenShift Service on AWS (ROSA). The ideal candidate should have hands-on experience in Kafka platform administration, cluster management, topic administration, connector development, and ongoing maintenance of existing integrations. The role requires a combination of platform engineering, development, operational support, and monitoring skills to ensure reliable, scalable, and highly available event-driven integration solutions.
Responsibilities
Administer, configure, and maintain Kafka platforms across on-premises, cloud, and Red Hat OpenShift Service on AWS (ROSA) environments. Manage Kafka clusters, brokers, topics, partitions, replication factors, retention policies, consumer groups, and access controls. Perform Kafka platform administration, including cluster health monitoring, capacity planning, performance tuning, and availability management. Deploy, configure, and support Kafka components and services across multiple environments, including development, testing, staging, and production. Develop, configure, deploy, and maintain Kafka connectors using Kafka Connect and related integration frameworks. Create current source and sink connectors to integrate Kafka with databases, enterprise applications, cloud services, and other external systems. Maintain and enhance existing connectors, troubleshoot connectivity and data ingestion issues, and implement fixes, upgrades, and performance improvements. Support connector lifecycle management, including configuration, deployment, validation, error handling, monitoring, and recovery. Manage Kafka topics and schemas,
including topic creation, configuration changes, schema compatibility, and data retention requirements. Work with Kafka Connect, connector plugins, serialization formats, and data transformation mechanisms to support reliable data movement. Use Kafka CLI commands and administrative tools to manage topics, consumer groups, configurations, offsets, ACLs, and cluster operations. Troubleshoot producer and consumer issues, consumer lag, broker failures, connector task failures, rebalancing, and message delivery problems. Configure and maintain monitoring, alerting, and dashboards using Datadog to track Kafka cluster health, throughput, latency, consumer lag, connector status, and infrastructure utilization. Develop and maintain Datadog monitors, alerts, dashboards, and incident notifications to proactively identify platform and integration issues. Use GitHub for source control, connector configuration management, code reviews, versioning, and collaboration across development and operations teams. Support CI/CD pipelines and automated deployment processes for Kafka configurations, connector changes, and application releases. Provide L2/L3 production support, participate in incident management, perform root cause analysis (RCA), and implement preventive measures. Maintain operational documentation, runbooks, standard operating procedures, and platform administration guidelines.
Collaborate with application developers, infrastructure teams, architects, security teams, and business stakeholders to deliver secure and reliable event-driven integration solutions. Follow best practices for Kafka security, platform governance, high availability, disaster recovery, and operational excellence.
Required Skills
Minimum 8 years of experience in middleware, messaging platforms, streaming technologies, platform administration, or integration development, with strong hands-on Kafka experience. Strong expertise in Apache Kafka administration and development across on-premises and cloud-based environments. Hands-on experience with Kafka on Red Hat OpenShift Service on AWS (ROSA), including deployment, configuration, operations, troubleshooting, and platform support. Experience administering Kafka clusters, brokers, topics, partitions, replication, retention settings, consumer groups, and access control lists (ACLs).
Strong understanding of Kafka architecture, including producers, consumers, brokers, partitions, offsets, replication, and consumer group management. Practical experience with Kafka Connect, including connector deployment, configuration, troubleshooting, and lifecycle management. Experience developing or customizing Kafka source and sink connectors and maintaining existing connectors in production environments. Strong knowledge of connector configuration parameters, plugin management, connector tasks, error handling, retries, dead-letter queues, and offset management. Proficiency in Kafka CLI commands and administrative utilities for topic management, consumer group inspection, configuration changes, ACL management, and troubleshooting. Experience with command-line tools and
📌 Kafka Platform Administrator (Calgary)
🏢 Tekfortune
📍 Calgary