Breaking the Bottleneck: Decentralizing Data to Power Library Innovation

Legacy designed library systems like hoopla challenged to meet risingexpectations for digital services and data-driven decisions. Built over adecade ago on outdated practices, Hoopla’s architecture has becomeinflexible, with data silos and complex integrations that limit real-timeanalytics and stifle innovation.

To modernize library services, we plan to redesign the architecture usingevent-driven design and stream processing. Our initial research suggeststhat adopting this approach will offer significant benefits for Hoopla. ApacheKafka will act as the event streaming backbone, facilitating real-time dataflow between modular library domains and systems. This method willenhance data accessibility, providing domain teams with real-time insightsand reducing integration complexities, thereby breaking down silos.

For stream governance, we plan to incorporate lineage tracking and acatalog for data discovery and quality. Flink will drive powerful analytics,enabling stateful stream processing and providing instant insights fromKafka event streams.

Beyond the technology, we’ll share our journey in data architecture,including:

- The critical mindset needed to transform rigid systems andunlock data insights

- The shift from centralized to flexible, stream-based architecture

- The impact on library operations, analytics, and next-gen services

As user expectations continue to rise, libraries must innovate with their datawhile managing complex integrations. This presentation distills ourexperience into architectural patterns and strategic recommendationsthat can be applied across sectors to master insights from data in motion.Attendees will gain a model for improving data flow, empowering domainteams, and achieving agile, analytics-driven systems.