The evolution of Notion’s event logging stack

Notion's client and server applications generate billions of events daily. We track these events to understand how our customers are using the product and what kind of performance they experience. Some events also contribute to customer-facing product features. This talk covers the Event Trail platform that enables us to process and route these events in a scalable manner.

Event logging at Notion was initially built on third-party services with connectors to our Snowflake data lake. This lacked the scalability and flexibility that we required as our product grew, and so we built Event Trail.

Event Trail receives events from the application, augments their content, and then directs them to one or more destinations based on their type. Routing is defined in code with dynamic overrides and honoring of cookie permissions. The most common destinations are Apache Kafka topics powered by Confluent.

The data warehouse ingestion pipelines read events from Kafka and write them to Snowflake. They were originally based on Apache Flink and S3 ingestion but have evolved to use Snowpipe Streaming connectors for easier maintenance and scalability. The real-time analytics pipelines use events to power user-facing features like page counters and enterprise workspace analytics. These features have also evolved, from batch results served via DynamoDB and Redis to online calculations via Apache Pinot.