Background

September 17-18, 2024 | Austin Convention Center

Keynotes
Breakout Sessions
Lightning Talks
Data Streaming in the Age of AI

Get ready to dive into the future of data! Jay Kreps and Confluent's top minds are set to reveal the next game-changing evolution of data infrastructure that effectively leverages AI and ubiquitous automation. Discover how leaders in professional services, media, and automotive industries are harnessing the power of Confluent's Data Streaming Platform and Data Products to revolutionize their operations. This is your chance to see how real-time data is driving innovation, transforming decisions, and propelling businesses into the future. The future is here—let's ignite it together!

Presenters
Presenters
Jay Kreps
Shaun Clowes
Kushal Khandelwal
Seema Acharya
Adam Tybor
Richard Artoul
The Rise of the Data Streaming Engineer

What does the rise of data streaming mean for software and data professionals? A once-in-a-generation paradigm shift is giving rise to a brand new role: the Data Streaming Engineer. Come hear from your fellow technologists how this new role is bringing together the work of application developers, data engineers, data scientists, and architects, and how it might describe the work you're already doing. Along the way, we'll get a survey of the current state of the technologies underlying the emerging Data Streaming Platform, and how you can prepare to use them in the coming year.

Presenters
Presenters
Matthew Seal
Tim Berglund
Adi Polak
Anna Povzner
David Anderson
Lucia Cerchie
Danica Fine
Sandon Jacob
Folks, THIS is Apache Kafka

In this session, let’s (re)introduce ourselves to the components of Apache Kafka. We’ll start with terminology like topics, partitions, producers and consumers - basic “blocking and tackling.” Along the way we’ll delve into how each is related and significant to the others. We’ll also take a high-level look at “special teams” aspects of the Apache Kafka ecosystem for data governance, connecting to external systems, and event processing.

Presenters
Sandon Jacobs
Presenters
3 Lessons We Learned Running Stateful Streaming Pipelines with Apache Flink and Kafka

At Bloomberg, we deal with tremendously high volumes of data. Financial markets move significantly from millisecond to millisecond. Clients must react to these immense amounts of market data and trust Bloomberg to synthesize and analyze the information in real time.

Presenters
Da Huo
Presenters
The SQL Ecosystem: Powering the Instant World with 40-Year-Old Legacy?

This talk is about the past, current state, and future of SQL. What were the key ingredients for its success? A declarative yet human readable syntax, automatic validation/optimization, catalog integration, schema-awareness, and built-in operators that get the job done. Great. Isn't that what we want in every API?

Presenters
Timo Walther
Presenters
Leveling Up Customer Experience with Kafka: From Modeling to Activation

Step into the gaming world where Kafka is the ultimate power-up for customer modeling and activation. In this session, we'll reveal how Kafka's event streaming capabilities are the secret weapon behind real-time insights and dynamic customer interactions. Discover how we've mastered Kafka to unlock significant business advantages and take our customer experience to the next level. Whether you're on a quest to upgrade your data infrastructure or enhance customer engagement, this talk will equip you with the strategies to leverage Kafka for a winning edge. Join us and learn how to conquer the game of customer experience with Kafka.

Presenters
Sanju Sriram
Presenters
Keeping your State at Bay: Patterns to Limit Kafka Streams Store Sizes

The delivery service startup OPS (Otter Parcel Service) is efficiently managing its fulfillment process. A team of skilled otters had written a small but mighty fleet of Kafka Streams topologies, and soon OPS was serving its first customers. All data streams ran smoothly, and operators and clients tracked parcel movements in real-time. Everyone was happy, and OPS was on the road to success.

Presenters
Hartmut Armbruster
Presenters
Speed Wins: The Basics of how to push more queries through each CPU core

When trying to scale Real-time Analytics projects, we usually focus on things like the infrastructure, the tech stack or the architecture, but we often forget about the basics: the things that can help build highly performant and low-latency queries for each specific use case.

Presenters
David Margulies
Presenters
Empowering Teams: Learning Strategies That Fuel Kafka Adoption

Forbes cited that 84% of companies fail at digital transformation. Harvard Business Review cited that 70% of all change initiatives fail. It may seem like the odds are stacked against leaders and organizations looking to make big moves in the digital space, but it is possible!

Presenters
Chad Layne
Cheri Augustine
Presenters
Consistency and Streaming: Why You Care Should About It

Streaming systems are often evaluated by a single metric: performance. We look for systems that can compute up-to-date results over volumes of changing data as promptly as possible. But if we stop our evaluation there, we ignore a pretty critical question: do the results we’re computing make any sense?

Presenters
Jessica Laughlin
Presenters
Leveraging LLMs for SQL Query Automation and Real-Time Data Streaming with Cardinal Health Sonexus, Google Cloud Vertex AI, and Confluent

In this breakout session, we will explore how to harness the power of Large Language Models (LLMs) to automate SQL query generation and seamlessly stream results into Confluent using Vertex AI. This session will offer an in-depth exploration of how to integrate advanced AI models into data pipelines, facilitating real-time analytics and decision-making in healthcare.Attendees will learn how to leverage LLMs to simplify SQL query creation, enhance data processing workflows, and utilize Vertex AI for scalable and efficient machine learning operations. Join us to discover innovative ways to optimize your data infrastructure and drive actionable insights with cutting-edge AI and data streaming technologies.

Presenters
Pascal Vantrepote
Merlin Yamssi
Steve Gregory
Presenters
The Architecture of Open Source Message Brokers

Kafka. RabbitMQ. Kinesis. NATS. Pulsar. And so many others. Which one should I use and why? This talk goes over the architecture of the different message brokers in depth, delves into their internals, and highlights the key tradeoffs to consider when choosing one over another.

Presenters
Veeral Patel
Presenters
Simplifying Apache Kafka®: Easy Configurations for Immediate Impact

In the realm of event-driven architecture and real-time data processing, Apache Kafka® stands at the forefront in providing an efficient and scalable solution for handling large volumes of data streams in real-time. However, developers often find interfacing with Apache Kafka® producers and consumers to be a labyrinth of complexities.

Presenters
Bethan Hughes
Niamh Thornbury
Presenters
Streamlining Entry into Streaming Analytics with JupyterHub and Apache Flink

The entry barrier into the world of streaming analytics often proves daunting, particularly for data scientists and machine learning engineers. The necessity of orchestrating multiple distributed systems can be a significant challenge. This obstacle is further compounded by the need to configure non-domain tools on local machines or Kubernetes clusters, before even initiating the development of streaming pipelines.

Presenters
Elkhan Dadashov
Presenters
Multi-Cloud Data Sharing: Make the Data Move for You Across CSPs Using Cluster Linking Description

Data sharing across cloud service providers has emerged as a mission-critical need for large-scale enterprises implementing cloud agnostic solutions. However, event driven business applications with the need to share data across CSPs face compounded complexities including network layer configurations, disaster recovery, and data governance.

Presenters
Eric Lipps
Michael Allen
Presenters
Scaling Data Ingestion: Overcoming Challenges with Cell Architecture

Ingesting massive volumes of data, often reaching hundreds of gigabytes per second, poses a significant challenge for companies across various industries, and New Relic is no exception. Leveraging cell architecture has proven to be an effective strategy for scaling New Relic's pipelines, yet it has introduced its own set of complexities. In this presentation, Ramesh and Anton will share invaluable insights gained from their experiences in overcoming these limitations.

Presenters
Antón Rodríguez
Ramesh Motaparthy
Presenters
Visualization in Motion: How to Create Effective Data Visualization with Real-Time Data

Is your data visualization optimized for your real-time data? Likely not. Every company needs a real-time data strategy but even when they have one, they often neglect to invest in charting solutions that can handle that data. It's easy enough to show throughput on a line chart or track offsets visually but are those the most effective methods for observing, analyzing, and diagnosing real-time data? We can do better.

Presenters
Elijah Meeks
Presenters
Otter Romp: Shallow Dives into Kafka Streams

What do you call a group of Kafka Streams developers? What do they spend their time on? And how can you get in on the fun? This talk will leave you with the answers — and hopefully an itching to jump in and get your feet wet too.

Presenters
A. Sophie Blee-Goldman
Presenters
Breaking the Bottleneck: Decentralizing Data to Power Library Innovation

Legacy designed library systems like hoopla challenged to meet rising expectations for digital services and data-driven decisions. Built over a decade ago on outdated practices, Hoopla’s architecture has become inflexible, with data silos and complex integrations that limit real-time analytics and stifle innovation.

Presenters
Mihir Agochiya
Ram Surabi
Presenters
Powering Real-Time Generative AI with Amazon Bedrock and Confluent Streaming

Generative AI thrives on real-time data access for relevant responses. This session covers integrating Confluent's scalable data streaming platform with Amazon Bedrock, a fully-managed Generative AI AWS service, to build innovative solutions. We will explore patterns for ingesting, processing, and serving data streams at scale and how to leverage Confluent's pre-built connectors, stream processing, and low-latency inference to contextualize AI models with real-time data and stream governance for trustworthy generative AI. Learn to create real-time knowledge bases, accelerate innovation cycles, and build secure AI applications. Enable real-time context, scalability, flexibility, and exceptional customer experiences.

Presenters
Weifan Liang
Pascal Vantrepote
Presenters
Real-time Platform Engineering: Scale Kafka Adoption Through a Kafka-as-APIs Program

Apache Kafka is an integral part of the data infrastructure of organizations aiming to build real-time applications, event-driven architectures, and microservices. However, the true potential of Kafka is only unlocked when Engineering teams across the organization are equipped with seamless access to building on top of Kafka data streams.

Presenters
Alex Drag
Presenters
Using Kafka Streams to Build a Data Pipeline for the Hospitality Industry

More than 12,000 hotels globally rely on Revinate’s Guest Data Platform and guest communication solutions to drive direct revenue and deliver delightful guest experiences.

Presenters
Daniel Brewer
Dean Burson
Presenters
Sentiment Analysis in Action with Apache Flink: Building Your Real-time Pipeline

Monitoring and interpreting sentiment of data records is important for a variety of use cases. However, traditional human-based methods fall short in handling huge volumes of information with required speed and efficiency. Yet AI can.

Presenters
Olena Kutsenko
Presenters
Introducing Apache Flink: 5 Things You Need to Know to Get Started

Apache Flink is one of the most popular stateful stream processing frameworks in the world. For those who haven't had a chance to explore it yet, we'd like to introduce the basics on what you need to know about Apache Flink in order to get started with your first streaming application.

Presenters
Nico Kruber
Presenters
Tapping into the Stream: Streamlining Real-time Data Acquisition and Analysis

Real-time information about customer and systems behavior offers a wealth of opportunities for modern companies, from enabling real-time, business-aware monitoring and reacting to the creation of novel customer experiences. Despite the consensus around this, tapping into real-time data sources is challenging for various reasons, e.g., different systems can use different technologies, different event backbones and heterogenous data formats. All these hinder the efficient sharing of real-time information which, in turn, reduces the ability to harness it.

Presenters
Marinos Kintis
Presenters
A Survey of Fake Data Generation Tools for Data Streaming Applications: Stop Writing Your Own

Dummy data is commonly used for testing applications. For small, functional tests, developers can quickly and easily mock up something close to what they expect. When it comes to data streaming applications, however, generating realistic streams of data that test all the functionality in an application means choosing from one of a crop of data stream generating tools, each with their own nuances, strengths, and limitations. So how does one choose?

Presenters
Afzal Mazhar
Presenters
Robinhood’s Kafka Journey from EC2 to Kubernetes

Robinhood relies heavily on Apache Kafka for many functions that power the application’s features. From stock trading to crypto, Kafka is involved in almost every mission-critical step of Robinhood’s functionality.

Presenters
Mun Yong Jang
Nathan Moderwell
Presenters
Data Contracts In Practice With Debezium and Apache Flink

Log-based change data capture (CDC) is an invaluable part of the data engineering toolbox: it enables a variety of use cases such as real-time analytics, full-text search, or cache invalidation by publishing data change events from your database. But when publishing change event streams across context or team boundaries, aren’t you tieing external consumers to your application’s data model, thus limiting yourself in evolving the same?

Presenters
Gunnar Morling
Presenters
Integrating MongoDB data into the Kafka and Confluent Cloud ecosystem through Atlas Stream Processing

Are you streaming data between MongoDB and Kafka and looking to enhance your setup? This talk will dive into various methods for integrations between MongoDB and the Kafka ecosystem. First, we’ll examine the traditional approach with Kafka Connect. Then, we’ll cover MongoDB’s new stream processing solution, Atlas Stream Processing. This fully managed service uses the document model and MongoDB Query API to offer a user-friendly, cost-effective, and high-performance way to handle your data. You’ll leave with a clear understanding of Atlas Stream Processing’s capabilities and how to apply them to your streaming needs with Kafka and Confluent Cloud.

Presenters
Robert Walters
Presenters
Tracking Locomotive to Train Associations in Kafka

A deep dive into the iterative design process of creating a highly resilient enterprise service for tracking locomotive associations for over a thousand trains all at once.

Presenters
Dean Schaefer
Presenters
Kafka - No Rocks, Please : Using Kafka Streams With Alternative State Stores

A popular data transformation library in the Kafka world - Kafka Streams - is bundled with a high-performance key/value store - RocksDB - with many advantages regarding near real-time, high throughput, and low latency processing. But what if RocksDB key/value store semantics are just not enough? What if the use case requires features of other storage engines - document indexing, graph processing, or simply multiple secondary indexes? Can we employ other embeddable storage engines, for example, Lucene, Neo4j, or extend RocksDB-based state stores?

Presenters
Roman Kolesnev
Presenters
Timing is Everything: Understanding Event-Time Processing in Flink SQL

In the stream processing context, event-time processing means the events are processed based on when the events occurred, rather than when the events are observed (processing-time) in the system. Apache Flink has a powerful framework for event-time processing, which plays a pivotal role in ensuring temporal order and result accuracy.

Presenters
Sharon Xie
Presenters
Streamlining History: ClickHouse & Flink's Fast-Track for Data Backfills

Integrating historical data with real-time streams is a common challenge in data streaming applications. Our approach leverages Apache Flink's Hybrid Source capabilities and ClickHouse's storage efficiency to selectively synchronize only the data that matters. This strategy streamlines the backfill process by leveraging true random access capabilities, unattainable with Kafka alone, dramatically reducing the time and resources required for effective data integration in streaming environments.

Presenters
Rafael Aguiar
Presenters
Modernizing Systems with Kafka Connect

In 2021, we presented “Kafka Connect for Near Real-time Data Moving Pipelines” at Kafka summit. Since then, Kafka Connect has evolved and matured, supporting businesses move data efficiently in near-real time. In today's rapidly evolving technological landscape, the need to migrate data from traditional databases to modern databases is important for business efficiency and staying competitive.

Presenters
Ashish Vijaywargiya
Sanath Shetty
Presenters
Taking Charge of Tables with OpenHouse

In the world of Big Data Management, there's always a tussle between control and flexibility! Cloud data warehouse solutions maintain tight control over the systems, which allows data infrastructure teams to ensure proper governance, data integrity, security, and performance. However, these systems often suffer from a lack of flexibility and scale. That's where open source data lake(house) systems come into play.

Presenters
Sumedh Sakdeo
Presenters
Self-Hosting Kafka at Scale: Netflix's Journey and Challenges

At Netflix, our usage of Kafka has significantly grown and evolved over the years, paralleling our product development and expansion. This talk aims to share our unique journey with Kafka, tracing our path from using it behind the scenes for building Keystone to now offering Kafka as a Service.

Presenters
Nick Mahilani
Piyush Goyal
Presenters
Test Data Streaming Applications With Confidence!

Testing real-time data streaming applications is hard but without robust integration tests you cannot ship your code with confidence. In this session we explore how Testcontainers libraries allow you programmatically create, manage lifecycle, and configure ephemeral instances of popular data steaming technologies. From spinning up individual Apache Kafka services to creating complex topologies your tests are in control of the environment they require. You can also run the same setup in your local development environment or in CI pipelines, which makes reproducing issues locally so much easier and fixes - faster. Testcontainers approach gives your flexibility and confidence to reliably test your code, and after this session you'll learn enough to start testing your new and existing data streaming applications.

Presenters
Oleg Šelajev
Presenters
Enabling Flink's Cloud-Native Future: Introducing Disaggregated State in Flink 2.0

In this presentation, I will introduce disaggregated state, a novel architecture we propose in Flink 2.0. It leverages Distributed File Systems (DFS) as the primary storage solution to address critical challenges faced by Flink in the cloud-native era

Presenters
Yuan Mei
Presenters
Kafka Streams at Scale: Avoiding Common Pitfalls

At LittleHorse, we use Kafka Streams extensively for a use-case that requires transactional semantics, high throughput, low latency, large amounts of state storage, and high availability. Seems like a tall ask, eh?

Presenters
Mateo Rojas
Presenters
Unlocking Real-Time Insights: Uber Freight's Evolution from Batch to Streaming Analytics

With Uber Freight overseeing a staggering billion loads annually, the demand for real-time analytics is paramount. Delve into our journey as we transitioned from traditional data aggregation methods reliant on stored procedures to a dynamic real-time application powered by Apache Kafka and Flink. Explore on how this innovative architecture revolutionized our operations, slashing data aggregation latency from fifteen minutes to mere seconds.

Presenters
Siddhi Talele
Thomas Bui
Presenters
Fortifying Your Transactions with KIP-890

Transactions are crucial to exactly once semantics in Apache Kafka®. But over the years we've found some gaps in the protocol. Enter KIP-890: Transactions Server-Side Defense! The goal of this KIP is to address some known issues like hanging transactions and to strengthen the protocol to make sure we write our messages not zero times or two times but exactly once! This session will cover the rationale of the KIP, the new changes introduced by the KIP, and future work made possible by the KIP. It will also cover how to set up your Kafka cluster with the improved transactional protocol.

Presenters
Justine Olshan
Presenters
Centralized Monitoring and Troubleshooting for Kafka Clients

Kafka clients collect a lot of metrics that can be used for troubleshooting. When you have ten clients in a cluster, it's already hard to coordinate collection of the metrics. When you have thousands of clients, it's really hard. Many cluster operators do not have control over the clients, and the lack of consistent telemetry across clients is an operational gap.

Presenters
Andrew Schofield
Apoorv Mittal
Presenters
A Tale of Two Platforms: Routing Events at the Edge

Embark on a transformative journey from legacy operations to cloud innovation. A small team began a mission that would evolve into the event-driven backbone of a global Enterprise, modernizing its architecture. We'll explore the construction of a robust event streaming platform that tackled two core challenges: interfacing with legacy systems constrained by limited hardware and isolated data silos and innovating a solution that allowed edge systems to run in the most remote locations without internet connectivity, ensuring seamless data integration with Kafka and our new cloud platform upon reconnection.

Presenters
Nic Pegg
Presenters
Understanding Stream Table Duality Using Kafka, Flink SQL and Debezium Format

Kafka Streams, ksqlDB or Flink SQL are popular processing engines that enable us to run SQL queries on top of streaming data. Isn't it fascinating that we can run SQL queries on top of streaming data as if they were relational tables, or convert a table into a stream of changelog events? This is known as the stream-table duality.

Presenters
Muhammet Orazov
Presenters
Vector Databases (pg_vector) and Real-Time Analytics

This session introduces the basics of vector indexes and vector databases, providing essential knowledge on their use. We will guide you through setting up a Real-Time RAG (Retrieval-Augmented Generation) data pipeline in Kafka and Pinot, a real-time OLAP and vector store. We'll demonstrate how to load documents from a website and perform question-answering on that content. I'll also talk about how AI and Real-Time analytics are converging. Key technical details covered include vector indexes, Apache Pinot, RAG, and creating embeddings for similarity searches. By the end of the session, you will understand how similarity search can be applied in real-time scenarios. This session is ideal for anyone interested in RAG, AI, vector databases, and real-time analytics.

Presenters
Hubert Dulay
Presenters
Powering Intelligent AI Apps with Confluent Cloud on Azure

Discover how the Native ISV Offering on Azure can simplify your AI and cloud projects with features like centralized billing, seamless integration, and automated data pipelines. In this session, we'll tackle common AI challenges, including data processing and transformation, and demonstrate how our tools can accelerate your work. We’ll wrap up with a live demo and share self-service video tutorials and hands-on code samples, so you can get started quickly with your Azure account. Perfect for developers and cloud architects looking to enhance their AI capabilities.

Presenters
Reshmi Sriram
Isreal Ekpo
Ankita Yadav
Presenters
Speed of Apache Pinot at the Cost of Cloud Object Storage with Tiered Storage in StarTree Cloud

For real-time analytics, you need systems that can provide ultra low latency (ms) and extremely high throughput (1000s of qps). One such system is Apache Pinot, which is excellent for real-time analytics use cases like user-facing analytics and personalization. The users of Pinot love the speed of Pinot and want to use Pinot for all their use cases - internal analytics, ad hoc analytics, reporting. For such use cases, you typically need to store really long retention data.

Presenters
Neha Pawar
Presenters
The Kafka Consumer: An Unexpected Journey of Data Consumption

Once your data is stored on your Apache Kafka® cluster, the next step is to consume that data and do something interesting with it. Enter: Kafka Consumers. We all know how to set up a Kafka Consumer to poll data… but do you know how a consumer fetches the data from the cluster? Let’s find out!

Presenters
Danica Fine
Presenters
Events at the (API) Horizon: How to synthesize domain events & changes from your HTTP/REST APIs

REST-based Request-response APIs are the lifeblood of most architectures. API transactions,specifically involving the CUDs (Creates, Updates, Deletes) are great sources of events and/orchange data. This is also true of various RPC style APIs.However, because of the client-server nature, such events need to be published by the service,using an "outboxing" mechanism and subsequently disseminated to pub-sub event broker,most often Apache Kafka. This involves code changes to the application and in addition, a moreexpansive solution like CDC (change-data-capture) to collect those changes from the outboxtable.

Presenters
Pavan Keshavamurthy
Presenters
From 📎 to 🧠: Building Gen AI Apps & Copilots on Streaming Data with Flink SQL

Gen AI apps such as intelligent business-specific Copilots can serve as a vital link between foundational models and data streaming, enhancing developer and employee productivity. Enabling users to ask questions about streaming data in natural language can provide business and engineering teams with dynamic, real-time access to organizational knowledge, enhancing employee productivity and improving workflows across enterprise operations.

Presenters
Steffen Hoellinger
Presenters
Protobuf Support in Uber's Real-Time Data Stack

A Real-time data platform (Apache Kafka, Apache Flink and Apache Pinot) plays a crucial role in Uber’s serving stack which primarily uses Avro as serialization format. At the same time, Protobuf is the primary data format at Uber for online applications and services. The discrepancy make engineers to take additional effort in order to convert the Protobuf data model into the Avro data model to tap into the real-time data stack.

Presenters
Yang Yang
Presenters
Conquering the Hard Bits of Event Processing with Restate & Kafka

We demonstrate how to use Kafka and Restate together to easily build event-driven apps with properties that are normally considered rather hard to do

Presenters
Stephan Ewen
Presenters
Highly Reliable Streaming Ingestion from Kafka to Iceberg Tables with Seamless Schema Evolution

Streaming data ingestion often has to deal with evolving data formats. Schema of incoming records can change over time as new attributes get added by producers. Even when schemas evolve or records are invalid, the stream processing should continue without interruptions. Graceful handling of schema changes and data errors is critical for zero-downtime in modern real time data platforms.

Presenters
Noritaka Sekiyama
Presenters
MirrorMaker 2’s Offset Translation Isn’t Exactly Once (and That’s Okay)

Is your streaming application ready for a cluster failover event? MirrorMaker 2 offers replication of topic data and consumer offsets between Kafka clusters, and allows applications to recover from the loss of a Kafka cluster. MirrorMaker2 also includes support for exactly-once data replication with KIP-618, preventing errors from duplicating data. But turning on exactly-once doesn’t prevent data from being re-delivered to your application when using translated consumer offsets. Why?

Presenters
Greg Harris
Presenters
Immediate Infinite Zoom: Revolutionizing Real-time Data Interaction with Imply

The data coming into Confluent and Apache Kafka® is fresh and highly dynamic. Traditionally, gaining real-time insights has been challenging, often requiring multiple tools and resulting in a fragmented experience as users switch between different levels of detail.Imply transforms this process by allowing users to instantly explore data at varying levels of detail—much like navigating a digital map. Users can seamlessly transition from high-level overviews, spotting trends and patterns, to detailed views that examine specific events closely. This capability preserves the full context of real-time data exploration, ensuring that insights remain intact as users shift perspectives.Join this session to discover how Imply is helping companies to build real-time analytics applications with immediate, infinite zoom capabilities. We’ll showcase real-world examples of companies using Imply to bridge the gap between strategic insights and detailed analysis, offering a unified solution for real-time analytics.

Presenters
Eric Tschetter
Presenters
Real-time Adaptive Controls for Kafka Consumers

Modern Kafka consumers are equipped with hundreds of tunables. There are a lot of these tunables such as worker pool sizes, autoscaling policies, throttlers and circuit breakers that directly effect the consumer resilience. Finding ideal initial values for these tunables requires deep technical expertise. Also, these workloads change over time, requiring regular effort to re-tune stale parameters. As a consequence, configuration errors have become a source of operational toil and one of the major causes of overload, cascading service and system failures across the industry. Consumers should aim to expose a minimal configuration surface by dynamically adjusting parameters based on observations. Praveen will provide a deep-dive into how CrowdStrike is using real-time Adaptive Controls(inspired from TCP congestion control) to dynamically tune these parameters for improved resiliency using real-time sampling of errors and latencies, removing the need for periodic adjustment. He will also discuss lessons learned deploying the feature to CrowdStrike’s massive production systems that handles multiple trillions of events per day without causing any incidents.

Presenters
Praveen Yedidi
Presenters
Generating Deterministic, Synthetic Workloads for Stream Processing

Do you want to kick the tires on Apache Flink®? Do you want to make sure it can perform well for your target use case? How can you do that?

Presenters
Michael Drogalis
Presenters
Modernized Trading: Managing Market Data with Confluent Cloud at Vanguard

Vanguard is one of the world’s most respected investment management companies, offering a broad selection of investments, advice, retirement services, and insights to individual investors, institutions, and financial professionals.

Presenters
James Tong
Matt Jordan
Presenters
Flinking Enrichment: Shouldn't This Be Easier?

A common pattern in stream processing applications is to enrich incoming events with contextual information. For example, a fraud detection pipeline might enrich incoming transactions with information about the customer, the merchant, previous transactions, etc, as part of building up a feature vector to hand off to a fraud detection model.

Presenters
David Anderson
Presenters
Pinterest’s Journey to an Automated, Efficient, and Low-Maintenance PubSub Platform

At Pinterest, Apache Kafka® runs on ~2000 brokers across 50+ clusters, moving data at petabyte scale every day. Through investing in automation and focusing on cost efficiency, we have seen positive results in dev velocity, scalability, stability, and cost footprint, enabling everyone to quickly and easily launch data streaming use cases that power business growth, while keeping our platform low-maintenance.

Presenters
Jeff Xiang
Vahid Hashemian
Presenters
Which Data Format Should I Use in Apache Kafka®?

You're just starting your data streaming journey or are extending it to include Schema Registry and you learned that multiple serialization formats are supported. How do you know which one(s) to choose? What if the choice is not yours? If you are offering the data streaming service as a platform to your internal customers, what kind of guidance should you offer them when choosing a data format? Is everything which is possible advisable? What implications does mixing data formats have? How do different formats differ from each other in terms of limitations, evolution and linking options? How does this affect consumers?

Presenters
Maria Berinde-Tampanariu
Presenters
CDC Pipelines to the Data Lakehouse Using Apache Hudi and Apache Flink

Change Data Capture (CDC) is crucial in capturing and disseminating real-time changes from upstream OLTP databases. Data Lakehouses provide scalability for data exploration on open data formats, but many struggle with record level updates in CDC workloads. Apache Hudi® has emerged as a comprehensive lakehouse platform purpose-built for managing mutable data, facilitating near real-time insertions, updates, and deletes with a robust incremental processing framework. Hudi’s integration with Apache Flink® bridges the gap between the streaming and data lakehouse communities, enabling low-latency, real-time data applications for various use cases such as change log reconcilation, ETL pipelines, streaming joins, and constructing materialized views.

Presenters
Shiyan Xu
Nadine Farah
Presenters
Unbundle Your Cake and Eat It Too: Redefining the Limits of Kafka Streams

Kafka Streams powers mission critical applications across a variety of use cases at some of the biggest companies in the world. What are the properties that make Kafka Streams so popular amongst developers looking to build event driven applications on top of Kafka? What are the operational risks associated with using Kafka Streams in your mission critical applications? Most importantly, what can you do about those risks?

Presenters
Apurva Mehta
Presenters
The Enterprise Data Supply Chain

In the era of digital transformation, effective data management is crucial for maintaining transparency, consistency, and governance. In their session, "The Enterprise Data Supply Chain," Adam Tybor and Tom Mahaffey, AI and Data Architects for Accenture Global IT, will presentAccenture's blueprint for managing data from source to consumption.They will highlight the importance of source-aligned data products reusable across various channels, including Generative AI and RAG models. Key principles include broadening data producers and consumers, leveraging metadata for automation and traceability, and utilizing cloud-native capabilities and real-time data movement for optimization.

Presenters
Adam Tybor
Tom Mahaffey
Presenters
Topic Federation: Enhance Kafka Availabilty with Sharded Topics Across Clusters

Powering various business verticals (e.g. ride sharing, eats, and delivery) at Uber, the Kafka messaging system is one of the most critical components of the real-time data infrastructure and its availability is of utmost importance. Intuitive approaches such as scaling (vertical or horizontal) out a cluster, and adding more replicas for topics can enhance Kafka availability to a certain degree. However, it fails to address the fundamental issue of cluster-level degradation.

Presenters
Qiushi Han
Xinli Shang
Presenters
The Joy of JARs (and Other Flink SQL Troubleshooting Tales)

Apache Flink® is a powerful thing to work with but, my, it does test one's patience sometimes ;-) Particularly if you're new to using it, understanding how to troubleshoot problems that arise is an important survival skill to acquire.

Presenters
Robin Moffatt
Presenters
Bridging the Kafka/Iceberg Divide

Apache Kafka® has long been the de-facto standard for managing and distributing data in motion, and Apache Iceberg has been on the rise in a similar role for data at rest. A common pattern is to collect streaming data from Kafka into Iceberg so that it can be efficiently queried. However, this process can be complex, expensive, and error-prone. It entails a sequence of connectors, streaming, and batch jobs. These data pipelines consist of multiple steps where each step needs to be manually configured, managed, and scaled. This also results in multiple copies of the data without a source of truth. As the scale grows, so does complexity, cost, and toil.

Presenters
John Roesler
Vasiliki Papavasileiou
Presenters
Real-Time Entity Resolution At Scale

Case management systems (insurance claims systems, benefits administration, claimsprocessing, etc.) are typically forms-based and have long processing times. Learn how real-timeentity resolution – the deduplication of similar data – can drastically help with creating acomprehensive view of your data. In case management, you’ll need to understand who filed forwhich benefits so that you can accurately assess fraud cases, eligibility requirements, and manyothers.

Presenters
Charles Jekal
Presenters
Billion Dollar Frauds - Unleash the Power of Confluent’s Streaming A.I. to Combat Voice Vishing

In the wake of a startling scam where a branch manager in Hong Kong was deceived into transferring $35 million by a deepfake voice, the imperative to fortify against such frauds has escalated. This incident underscores the critical need for advanced defences against voice vishing, particularly as criminals leverage easily accessible generative A.I. to create highly realistic cloned voices. In response, we're running the inference of our Streaming Voice Vishing A.I. model in Confluent and Flink and directly integrating with call center solutions like Genesys to offer Fortune 500 companies a robust shield against these ferocious threats with billions of dollars in losses globally.

Presenters
Chris McMahon
Hussain Jaber
Presenters
Mastering Kafka at Scale: Unleashing the Power of Temporal at Stripe

Stripe is at the forefront of the payment processing game, handling an astonishing 700 terabytes in Kafka publish throughput daily – a feat that is only possible with the open-source powerhouse Kafka. However, managing 50 Kafka clusters across three environments and multiple regions can be a Herculean task. That's where Stripe's Stream Infrastructure team stepped in, leveraging the incredible capabilities of Temporal to build a state-of-the-art Kafka Control Plane.

Presenters
Pat Stefanou
Presenters
Optimizing Apache Kafka with GraalVM: Faster, Leaner, and Performant

Docker, a widely adopted container runtime, has revolutionised the way we deploy applications. With the implementation of KIP-975, Apache Kafka 3.7.0 and subsequent versions now come with a docker image.

Presenters
Krishna Agarwal
Vedarth Sharma
Presenters
Events and Entities, a Dichotomy for the Streaming World

The dichotomy of OLTP versus OLAP is very well known, but perhaps less well known is that the terms were coined circa 1993, i.e. it's only 30 years old. About 15 years ago, NoSQL was all the rage, as the boom of the internet meant that technology companies all started fighting trying to get horizontal scale out of systems built for vertical scale. This has given way to a wide expanse (perhaps it's even fair to call it an overwhelming expanse) of technology choices: Key-Value Stores, Document Stores, Search Engines, Streaming PubSub, Stream Processors, HTAP, Real-time Analytics, Cloud Data Warehouses, etc. etc.

Presenters
Eric Tschetter
Presenters
Beyond Tiered Storage: Deep dive into WarpStream's Storage Engine

In this talk, we'll discuss in technical detail how to take the concept of tiered storage to its logical extreme by building an Apache Kafka® protocol compatible system that has zero local disks.In this talk we'll cover how to design a storage engine from the ground up around commodity object storage.

Presenters
Richie Artoul
Presenters
Atomic Dual-write Recipes with Kafka Two Phase Commit (KIP-939)

Kafka is the central nervous system of organizational data that connects heterogenous data sources and sinks together. KIP-98 introduced exactly-once semantics for data processing within Kafka. KIP-939 brings exactly-once semantics to the next level and enables exactly-once semantics between Kafka and a database.

Presenters
Artem Livshits
Presenters
Unlocking the Power of Flink's Table API: A Deep Dive into Advanced Capabilities and Use Cases

In today's fast-paced digital world, the ability to process and analyze real-time data streams sets the foundation for innovative applications across industries. Apache Flink's Table API enables developers to write complex data processing queries in a simple, declarative manner using Java, Python or even combined with SQL. This talk aims to unravel all the capabilities of the Flink's Table API, providing attendees with a thorough understanding of its architecture, functionalities, and the immense possibilities it unlocks for real-time data analytics.

Presenters
Martijn Visser
Presenters
Change Data Capture & Kafka: How Slack Transitioned to CDC with Debezium & Kafka Connect

Change Data Capture (CDC) has emerged as a critical component in modern data processing architectures, enabling organizations to capture and process real-time data changes efficiently. This talk presents a detailed case study of Slack's transition to CDC, highlighting the utilization of Vitess, Kafka, Debezium, and Kafka Connect in revolutionizing data processing workflows.

Presenters
Joseph Thaidigsman
Tom Thornton
Presenters
Streaming Queries Without Compromise

Modern databases excel at analysis when data changes infrequently. However, for rapidly changing, many custom systems have been built under the guise of "streaming systems". To offer near-real time answers, streaming systems compromise on the expressiveness of the computations they perform.

Presenters
Mihai Budiu
Presenters
Atlassian's Lithium Platform: Dynamic, Self Hosted, and Distributed Ephemeral Streaming Pipelines

Atlassian has numerous use cases that require moving large amounts of data around, validating and transforming it in flight. Platforms such as Apache Flink can be excellent choices for moving and transforming data at scale - effectively streaming ETL. However our uses cases present some challenges - they require stream processing pipelines to be entirely provisioned at runtime, including dedicated Kafka topics, parallelism, selection of stream processors with appropriate and available compute resources, and our stream processors must be hostable directly in product and platform services to enable in-process access to service context in order to meet throughput and concurrency goals. In addition, our pipelines require coordination amongst sources, transforms, and sinks that exist in different product and infrastructure services. This led us to build the Lithium Platform - designed to meet these needs. It is 100% event driven and is built on Kafka and Kafka Streams.

Presenters
Robert Englander
Presenters
Apache Kafka's Common Pitfalls & Intricacies: A Customer Support Perspective

As Apache Kafka gains widespread adoption, an increasing number of people face its pitfalls. Despite completing courses and reading documentation, many encounter hurdles navigating Kafka's subtle complexities.

Presenters
Aurelie Marcuzzo
Christoph Schubert
Presenters
Effective Data Lineage Strategies for Real-Time Systems

Data in motion is data on a journey, and understanding that journey is a crucial part of streaming governance. Data lineage enables organizations to unravel the intricacies of real-time data flows, enhance transparency, and build trust across an event-driven landscape. Data lands on a topic, consumed and transformed, then produced to another topic, repeat. In this way, data travels through any number of topics and intermediary systems in real-time before reaching its final destination. This poses a significant hurdle; without detailed message-level tracking, chain of custody and transformations done on that data can be lost.

Presenters
Mark Soule
Presenters
Building a Scalable Flink Platform: A Tale of 15,000 Jobs at Netflix

At Netflix, we operate over 15,000 Apache Flink jobs, processing over 60 PB of data per day. Apache Flink is widely used across many engineering organizations within Netflix to power stream processing use cases such as data movement, personalization, messaging, and finance. This diverse usage of Flink manifests itself in different job characteristics, such as cluster size (up to 2000 parallelism), state size (stateless to 4 TB), and complexity of job graphs.

Presenters
Mark Cho
Mingliang Liu
Presenters
The Performance of Kafka's New Consumer Rebalance Protocol

KIP-848 (https://cwiki.apache.org/confluence/display/KAFKA/KIP-848%3A+The+Next+Generation+of+the+Consumer+Rebalance+Protocol) proposed a new paradigm of the rebalance protocol for Kafka consumers. This prompted a large architectural change in the group coordinator, primarily moving to an event-driven architecture.

Presenters
David Jacot
Jeff Kim
Presenters
Kafka Streams as a Data Store for a Workflow Engine

At LittleHorse, we are building a high-performance workflow engine for microservice orchestration. We have a long list of performance-related requirements, including low latency, horizontal scalability for throughput, transactional semantics, and high availability. Long story short: we decided to build our own data store on top of Kafka Streams.

Presenters
Colt McNealy
Presenters
How the World's Largest Options Clearinghouse Moved From Batch to Stream Processing to Manage Risk

In this session, we will review how the OCC performs complex analytics using Kafka and Flink on a data stream containing prices for over 2 million financial instruments per second and billions of open option positions. We will also review how Ness and the OCC satisfied the SEC's requirement to build isolated Kafka + Flink infrastructure in both AWS and an on-premises data-center for disaster recovery purposes. Using Kafka + Flink, the OCC radically improved performance for their core margin calculation system, going from an 8+ hour, overnight batch process, to a 5 minute intraday operation.

Presenters
Subir Grewal
John Martino
Vadim Gelman
Presenters
Addressing Streaming ETL Pipelines Challenges: Delving into Flink CDC

Data freshness significantly impacts the value of data insights, particularly for business data housed within databases. Establishing a more real-time synchronization pipeline is crucial for deriving actionable insights from this data. However, constructing such a pipeline encounters various challenges. Common issues include frequent table schema changes in the upstream database and the need to add or delete tables according to evolving business requirements. Moreover, flexibly scaling resources during historical reading and log reading stages proves difficult, while synchronizing multiple tables or the entire database consumes substantial resources. Additionally, constructing job workflows using the DataStream API poses its own complexities.

Presenters
Leonard Xu
Presenters
Kafka Unlocks Scalable Vector Search in Milvus

Powered by the popularity of ChatGPT and other large autoregressive language models, we've seen a huge surge in interest for vector search in 2023. In applications that leverage [RAG](https://zilliz.com/use-cases/llm-retrieval-augmented-generation), the Milvus vector database is commonly used as a way to retrieve relevant document chucks and other short-form text. The applicability of text-based RAG has led to widespread adoption, from single-person tech startups to Fortune 500 fintech companies.

Presenters
Frank Liu
Presenters
Trust and Safety at Indeed: Detecting and Preventing Fraud in Real-time

At Indeed, we help people get jobs. The Trust & Safety team at Indeed contributes to our mission by developing, enforcing, and advocating for quality, creates policies to govern content, and evaluates many different fraud vectors including job postings, messaging, data export, and more. This ensures the safety of jobseekers and that all employers are able to have a robust pool of qualified applicants to meet their hiring needs.

Presenters
Gayathri Veale
Ryan Merriman
Presenters
Getting Gory with Active-Active Multi-Region Disaster Recovery

Kafka is one of the most durable and reliable data systems in existence. Thousands of mission critical businesses have been ironing out the wrinkles over the last decade. Once you move your event driven apps or streaming pipelines out of the confines of a single region the route is not that well understood. Asynchronous replication and eventual consistency add several challenges irrespective of the technology you are using. Businesses have a strong desire to employ an active-active strategy to gain the benefits of data locality and to mitigate the blast radius of regional failures. When you sum this all up, you have yourself a complex challenge. In this talk, we will cover:

Presenters
Julie Wiederhold
Will LaForest
Presenters
CLI Tools You Need to Know for Your Everyday Kafka

Apache Kafka is all over the place! Now you can begin using brokers, topics, and clusters. If you, like many other users, prefer the terminal to the graphical interface or web consoles, you should be familiar with these CLI tools to increase your productivity. It is a collection of applications that can assist you with everything from creating a cluster to managing your Kafka Connect connectors or Kafka users.

Presenters
Hugo Guerrero
Presenters
Robinhood’s Flink Deployment Practices

Robinhood leverages Flink for many of the stream processing applications that power the platform’s features, including shareholder’s positions, fraud detection, data ingestion, etc. With Kafka involved in almost every mission-critical step of Robinhood’s functionality, Flink is crucial for building scalable, maintainable and reliable stream processing applications.

Presenters
Tony Chen
Presenters
Client Cannot Communicate - Lessons From a Decade of Kafka Consulting

We rightly celebrate the successes of high-profile Kafka projects, but what about the less visible failures? Let's delve into the invaluable lessons learned from these disappointments. Allow me to unveil the hidden truths of avoidable failures—strategies that can compromise your project's success, sabotage your engineering endeavors, and subtly diminish your team's productivity. Drawing from extensive experience across numerous Kafka projects in various sectors such as banking, transportation, retail and product development, I've collected my fair set of scars and lessons from the field that everyone should know

Presenters
Simon Aubury
Presenters
Disaster-Proof Data: Navigating Real-Time Streaming with Zero Downtime

Dive into the realm of zero downtime for real-time data streaming apps with a focus on Disaster Recovery best practices. Join me as I share practical insights and experiences from real-world production use cases that leverage multi-regional clusters. Discover how to architect resilient event-driven systems and ensure seamless operation even in the face of unforeseen challenges. Unleash the power of continuous data streaming with confidence!

Presenters
Niyi Odumosu
Presenters
Debezium vs. the World: An Overview of the CDC Ecosystem

Change Data Capture (CDC) has become a commodity in data engineering, much in part due to the ever-rising success of Debezium [1]. But is that all there is? In this lightning talk, we’ll outline the current state of the CDC ecosystem, and understand why adopting a Debezium alternative is still a hard sell. If you’ve ever wondered what else is out there, but can’t keep up with the sprawling of new tools in the ecosystem; we’ll wrap it up for you!

Presenters
Marta Paes
Presenters
Streaming Entity Resolution for Kafka with Quine

"These two records are really the same thing!" Whether it's misspelled company names, or users who sign up twice with different email addresses, or bank accounts controlled by the same shadowy figure, the data received in a Kafka stream doesn't always resolve to reality we're trying to represent. This problem has been slowing down streaming adoption across the industry—from the world's largest banks, to the small teams trying to deploy their first streaming data pipeline. But cleaning that data quickly has gotten easier recently with new open source streaming graph tools which can perform powerful entity resolution on streaming data at scale while it's still in motion, and even if it's out of order.

Presenters
Ryan Wright
Presenters
Streaming Analytics vs. Streaming Database - An Unbiased Comparison

Event stream infrastructure has become an important part of modern data stack in many organizations enabling them to rapidly ingest and process streaming data to gain real-time insight and take actions accordingly. Streaming storage systems such as Apache Kafka provide a storage layer to ingest and make data streams available in real-time. Streaming analytics(stream processing) systems and streaming databases are two platforms enabling users to gain real-time insight from their data streams. However, each of these systems is suitable for different use cases and one system alone may not be the right choice for all of your stream processing needs. In this talk we introduce both streaming analytics(also known as stream processing) and streaming databases and provide an overview of capabilities in each system. We also explore different categories of use cases and discuss which system is more appropriate for these use cases.

Presenters
Hojjat Jafarpour
Presenters
Breaking Kafka Immutability: Are you mad?

Who hasn't had to reset the offset of a consumer group to skip a defective record? You must stop your applications, identify the malformed records, adjust the offsets of the consumer groups reading this topic using some tools, and then restart everything. That's quite cumbersome, right?Have you ever dreamed of breaking Apache Kafka's immutability to delete or fix a record instead of juggling consumer offsets? What if I tell you that it's possible?

Presenters
Florent Ramiere
Presenters
So You Want to Write a User-Defined Function (UDF) for Flink?

Apache Flink gained a lot of momentum during the past couple of years and is well-known across the data industry for its versatility. Developers and data engineers alike preferably pick higher-level programming abstractions such as Flink's Table API and Flink SQL whenever reasonably possible. And yet, specific requirements might cause the need for writing custom code and integrating with 3rd party libraries - Flink UDFs to the rescue!

Presenters
Hans-Peter Grahsl
Presenters
Data State to Business Events: Application of Set Theory to Event Streaming

At ACERTUS, we are using Apache Kafka to deliver real business value to our customers by automating order lifecycle and providing near-real-time business events. One of the tools we created is an elegant architecture for collaboration between business operations and software developers to efficiently transform data events into business events.

Presenters
Duncan Werner
Presenters
Is Your Kafka 'Truly' Load Balancing? Pro Tips for Leader Partition Distribution with Network Usage

One of the primary goals of distributing leader partitions across all brokers in a Kafka cluster is load balancing. This practice helps prevent overloading any single broker and optimizes the throughput of the entire cluster. The distribution of these leader partitions is mainly determined by the preferred leaders set during the creation of topic partitions.

Presenters
Jungwook Song
Presenters
The Availa-blitz: Kafka's Scaling Saga for High-Speed Trading

In the fast-paced world of high-speed trading, every millisecond counts. Join us for a lightning talk as we unveil the remarkable journey of scaling Kafka for high-speed trading at Charles Schwab. Discover how Kafka's event streaming architecture revolutionizes real-time data processing, enabling Charles Schwab to handle 2 million messages per second with millisecond latency.

Presenters
Samuel Tjokrosoesilo
Presenters
SketchDB: Reducing Your AWS Analytics Cost by 10x Using Principled Approximation

Streaming analytics serve a variety of downstream applications. With rising volumes of data and an increased need for real-time analytics, analytics costs are ballooning out of control. Many downstream applications are amenable to working with approximate analytics and thus, methods like sampling are commonly used by practitioners to reduce costs. Unfortunately, in doing so, practitioners are faced with a cost-accuracy dichotomy, having to choose between low cost and high accuracy.

Presenters
Milind Srivastava
Presenters
Your Swiss Army Knife for Kafka-based Applications

Imagine you need to prove that a certain record is actually present in a topic. Or you need to quickly copy one topic to another cluster. Or you need to fix a corrupted topic where the schema information is missing. Or one of your colleagues from analytics would like to have a topic in Parquet format, or a manager in Excel... To do these jobs, so far, you would often resort to scripting with the existing command-line tools from the Apache Kafka distribution, or, in more complicated cases, to writing your own Java or Python application. This can be so tedious and time-intensive that it can bring entire Kafka-based projects down. In this lightning talk, you will be thrown into the midst of a typical Kafka nightmare of corrupt topics and schemas, where the deadline for the qualification test looms and there is no time to spare. And we will show you how Kafi, your Open Source Swiss army knife for Kafka, can help you to get out of this horror trip quickly and easily, with just a few lines of Python code.

Presenters
Ralph Debusmann
Presenters
Treat Data Like APIs

Microservices encapsulate code and data of a domain and establish standard interfaces to access the domain via APIs, but service owners sometimes forget that service data goes into a data lake for analysis or reporting. Traditionally, this data lands in a data lake via change data capture (CDC), and data teams use transformation jobs on this “raw” data to build curated datasets. But this breaks the microservice encapsulation and exposes internal data layout to external offline consumers. Making changes to the database for microservice becomes hard because numerous other data pipelines depend on it downstream. Data bugs are only discovered days after they first occur, because those data assets are separated by a couple of degrees from the team that owns the source of truth microservice database schemas.

Presenters
Gaurav Tungatkar
Presenters
“Failure to Connect” Bugs and Issues in Kafka Connect to Watch Out for a Healthier Environment

Kafka Connect is a great product at connecting systems together and ensuring messages are produced and consumed from topics to endpoints. In order to have the highest availability on Connect, you need to understand the “failures to connect” in regard to any bugs and issues to watch out for in Kafka connect.

Presenters
Catalin Pop
Presenters
Terms & Conditions Privacy PolicyDo Not Sell My InformationModern Slavery PolicyCookie Settings
Previous Years:
2022 Sessions2023 Sessions2024 Sessions
Copyright © Confluent, Inc. 2014-2024. Apache, Apache Kafka, Kafka, Apache Flink, Flink, and associated open source project names are trademarks of the Apache Software Foundation