From “Where’s My Money?” to “Here’s Your Bill”: Demystifying Kafka Chargebacks and Showbacks

Have you ever wondered how the money you spent on those kafka clusters is being utilized? Or how much end users should be paying for those awesome use cases that they run in production at scale without worrying about downtime and resiliency? How do you charge that one person who requested those 1000 partition topics, or the one who has like 3 out of 1200 topics but is using about 70% of the available network throughput for your cluster. If you ever wondered about any of these questions, this talk is for you.

In this talk, we will deep dive into ways to dissect your Kafka bills and attribute them to your end users, your business teams, your application teams that depend on these Kafka clusters. I will help you understand the fundamentals of how to approach chargebacks/showbacks for Kafka and show you how deep the rabbit hole goes. Using open source tooling and an example, we’ll discuss:

* Techniques to define a core identity – an mTLS certificate or a SASL user or a Business unit? Which one is it and which one should it be?

* How to envision cost split – Should it be spread evenly or should there be a usage based differentiation for things like network over-utilization? Noisy neighbour anyone?

* Chargeback – What should be the final output product of your process and how should it be delivered? Is an excel sheet enough or do you want a dashboard that keeps updating itself automagically?

By the end of this talk, you will be able to understand the fundamentals to help you either build out your own cost analysis for Kafka or use the tool to just say - “Here’s your Bill”.