What Can You Do with a (Kafka) Queue?

The “traditional” consumer group coordination in Apache Kafka assigns each partition of a topic to a member of a consumer group, providing a powerful combination of ordering and scalability. Sometimes ordering is not of the essence, and we would rather treat events as individual units of work.

Enter KIP-932 - aka “Queues for Kafka” - enabling multiple consumers in the group to process from the same topic-partition.

Every time I have spoken on this topic, I get the same questions: What are some use cases and when do I use it? So let’s take some time to identify and explore a couple of use cases. We’ll walk through code samples for these scenarios and how we can validate the behavior.

When we wrap up, you’ll have a better idea of how and where to get started using queues the Kafka way.