Apache Kafka 4.0.0 Released: KRaft, Queues, Better Rebalance Performance

Introduction to Kafka 4.0

Apache Kafka reached a major milestone in March 2025 with the release of version 4.0.0. This release introduces new features, improvements, deprecations, and important updates. Let’s dive into the highlights of Apache Kafka 4.0.0 and see how these changes affect developers, system architects, and integrators.

ZooKeeper is Gone: Long Live KRaft

For many years, running Apache Kafka required an external ZooKeeper cluster to manage cluster metadata and perform e.g. leader election. Such additional components added operational complexity both in production and during local development. That era is now officially over.

KIP-500 - was introduced as an early access feature in Kafka 2.8.0, released in 2021 – already four years ago! Over the following releases, KRaft (Kafka Raft) matured, gained production readiness, and introduced migration features that made it suitable for real-world use.

With Kafka 4.0.0, ZooKeeper support has been completely removed. KRaft is now the default – and only – way to manage metadata in Kafka. This significantly changes how Kafka clusters are designed and operated.

If you're migrating from a ZooKeeper-backed cluster to a KRaft-based one, be sure to review the official migration guide to understand the limitations, tooling, and best practices.

KIP-848: Smarter, Faster Rebalances With the New Consumer Group Protocol

KIP-848 introduces the next generation of Kafka’s consumer rebalance protocol, fundamentally changing how consumer groups assign partitions.

This improvement is crucial for large-scale deployments where application redeployments or consumer restarts frequently trigger rebalances, affecting system performance by halting message processing until the entire rebalance round finishes.

What changed?

• The broker now manages and assigns partitions, taking over responsibilities previously held by the consumer group leader.
• The Kafka client’s internal logic is greatly simplified.
• When a new member joins a consumer group, it can incrementally get new partitions. Before, it needed to wait for all members to revoke/rebalance their partitions which unnecessarily impacted consumers’ latency.

The new protocol is designed to dramatically improve performance, reduce downtime and latency, especially in large deployments.

Watch also the Confluent presentation on this topic from Confluent Current 2024.

KIP-932: Queues for Apache Kafka 4.0 – Early Access Preview

Kafka messaging model based on topics and partitions has become a cornerstone of modern event-driven architectures. However, some use cases require more traditional queue semantics, as seen in systems like RabbitMQ or Google Cloud Pub/Sub. Out-of-order processing becomes useful when processing of one of the messages may temporarily fail and needs to be retried. Lots of custom and complicated retry mechanisms have been built by teams over the years to address these requirements.

KIP-932 introduces a preview of Queues concept in Kafka 4.0, enabling new types of workloads that were previously not possible.

What it adds:

• The concept of shared groups, enabling cooperative consumption of messages.
• Support for individual message acknowledgments and redeliveries.
• Unordered processing, useful when some messages may temporarily fail and require retries.

This is a major step forward in Kafka's evolution, making it a viable choice for use cases that previously required a separate queuing system.

Want to dig deeper? Check out Adam Warski's blog post on Kafka Queues, where we explore this feature in more detail.

Deprecated APIs Retired and Updated Java Version Requirements

Major releases are an opportunity for cleanup, and Kafka 4.0.0 uses this chance to remove deprecated APIs and modernize the platform.

Notable Updates:

• Java 8 support has been dropped.
• Kafka Clients and Streams now require Java 11+.
• Kafka Brokers, Connect, and Tools require Java 17+.

This aligns Kafka with modern Java runtimes that offer improved performance, GC algorithms, and long-term support. With Java 11 available since 2018 and Java 17 since 2021, these versions are widely adopted and stable.

Another critical change: old protocol API versions have been removed. Kafka 4.0 Java clients now require brokers running version 2.1 or newer. Additionally, support for message formats v0 and v1 has been dropped, which may impact older producers or consumers.

Summary

KRaft, the new rebalance protocol, and Kafka Queues are three transformative changes in Kafka 4.0. It’s great to see the platform actively evolving, with a clear focus on simplicity, performance, and new use cases.

On the commercial side, we can expect Confluent Platform 8.0 (based on Kafka 4.0) to be released in summer 2025, likely bringing many of these features into enterprise-ready offerings.

Need help planning your Kafka 4.0 upgrade or exploring Kafka Queues?
As a Confluent Premium Partner, our team at SoftwareMill has deep Kafka expertise and can support your journey – from migration and tuning to architecture design.

👉 Let’s build scalable, event-driven systems together.

Reviewed by: Michał Ostruszka