Native Data Lineage Support in Apache Flink with OpenLineage

Apache Flink has made significant strides in native data lineage support, which is essential for auditing, data governance, regulatory compliance, troubleshooting, and data discovery. In this presentation, we will delve into Flink's built-in lineage graph and listener mechanism, showcasing its current capabilities and recent enhancements brought by FLIP-314.

We will emphasize how Flink's native lineage features provide a robust framework for understanding and managing data flows within streaming applications. Furthermore, we will explore the integration of Flink lineage with OpenLineage, an open framework designed for the systematic collection and analysis of data lineage. This integration facilitates seamless lineage data management and visualization across modern data ecosystems.

Join us to gain insights into the advancements of native lineage support within Apache Flink and learn how it can significantly enhance your data operations and compliance initiatives.