Design with me : a Kafka Streams Payment Authorization Collaborative Design Session

Most Kafka Streams tutorials end where production begins. This interactive, detailed-design, whiteboarding session bridges that gap by designing a real-world payment authorization system: collaboratively, honestly, and without hiding the messy parts.

We will start optimistically with Kafka Streams DSL approaches: elegant joins, straightforward windowing: and systematically discover their breaking points.

As Apache Kafka Streams DSL solutions fail under real-world constraints, we'll architect production-grade alternatives using the Processor API, custom state store designs, and advanced patterns for handling late data, implementing timeouts, managing state lifecycle, and ensuring exactly-once semantics. We'll visually map out state management strategies, “watermarking” approaches, and the trade-offs between different join patterns when dealing with temporal uncertainty.

Here’s the challenge (Inspired from one of my past engagements) ! Two asynchronous input streams:

shiny new payment authorization requests arriving in real-timeaccount state updates that arrive delayed, because that is how the legacy integration actually works. The business requirement is straightforward: decide whether to authorize each payment. The technical reality? Anything but simple.

This is not a lecture. Armed with a (virtual) whiteboard, we'll design the system architecture together (i’ll fill in the blanks !) progressively integrating new constraints that turn textbook examples into production systems: late-arriving state, authorization timeouts, authorized user override, state store growth.

Attendees will leave with mental models for recognizing when business requirements exceed DSL capabilities, practical patterns for designing custom state stores that match their domain logic, and confidence in choosing between DSL and Processor API approaches. Most importantly: the architectural judgment I've earned through production failures—so you can succeed without repeating them.