Deep dive into writing Queues for Kafka applications

Queues for Kafka introduces a new paradigm for consuming data from Apache Kafka, along with the new Share Consumer API. If you’ve ever struggled with building message-queuing applications on top of Kafka, struggle no more. The Share Consumer API in Apache Kafka 4.2 makes this easy.

The Share Consumer API still consumes data from Kafka topics, but if you think about how it works, it behaves much more like a message queue. Learn all about how partitions are shared by the consumers, taking precise control over record fetching, the ways of acknowledging record delivery properly, error handling strategies, and even how best to deal with records which take a very long time to process. If you’ve wondered how to read records from Kafka and process them with a group of workers using generative AI without worrying about head-of-line blocking, this is the talk for you.