Ever wonder what is the secret behind the legendary data-driven cultures of companies like LinkedIn, Airbnb and others? The answer is metadata!
In this session, Shirshanka Das presents the multiple ways in which metadata platforms shaped the data ecosystems at these companies, through his own journey in creating the open-source DataHub project. This project is now in wide use at companies like LinkedIn, Expedia, Saxo Bank, Klarna, Wolt and many others and is part of the critical day to day workflows of thousands of data professionals around the world.
We will deep dive into the product and architecture of DataHub, which includes capabilities like stream processing, schema-first extensible modeling and time-series metadata. These features allow multiple use-cases like discovery, observability, automated governance to be implemented.
The talk will describe how different companies have implemented these diverse use-cases on top of the same metadata platform, and how DataHub allows integration of multiple tools in the modern data stack to meet the demands of these use-cases. The talk will conclude with a short demonstration of how you can get started with DataHub and accomplish interesting things within 10 minutes or less.