Deploying Apache Kafka 4.2.0 on Kubernetes with KRaft, SASL, and High Availability

Overview This guide walks through deploying a production-grade Apache Kafka cluster on Kubernetes using KRaft mode (no ZooKeeper), SASL/SCRAM-SHA-512 authentication, and a 3-node StatefulSet for high availability. It covers every manifest file required, the reasoning behind each configuration decision, the SCRAM credential bootstrap process, common pitfalls encountered in practice, and the steps needed to take the cluster from running to production-ready. Prerequisites Tools kubectl configured against your target cluster A Kubernetes cluster with at least 3 nodes (one per Kafka pod) with sufficient resources Persistent volume provisioner available (e.g. local-path, Longhorn, Ceph, AWS EBS) keytool (part of the JDK) if you plan to add TLS later Cluster Resources Each Kafka pod in this guide requests 500m CPU and 1Gi RAM, with limits of 2 CPU and 4Gi RAM. For production you should size these based on your throughput requirements. A minimum of 10Gi persistent storage per broker is configu