Consistency and Streaming: Why You Care Should About It

Streaming systems are often evaluated by a single metric: performance. We look for systems that can compute up-to-date results over volumes of changing data as promptly as possible. But if we stop our evaluation there, we ignore a pretty critical question: do the results we’re computing make any sense?  

  Low-latency results often come at the expense of weak consistency guarantees. A popular consistency guarantee for streaming systems is eventual consistency: if the inputs stop changing, the output will eventually arrive at the correct result. But some use cases require stronger guarantees and always-correct results, like: automatically triggering user notifications, detecting fraud, or writing the title of your talk.    

 Fortunately, there are stronger consistency models we can layer over streaming systems, all the way up to the strongest of guarantees: strict serializability! In this talk, we’ll recap the levels of consistency guarantees available in streaming, and explain how the tradeoffs you make can make or break the products you build.