Technical Talks

When processing unbounded data sets (aka streaming), it is typical to enrich them with secondary data sets at some point. For example, given user activity it's common to join to user profiles. This could be for building a richer stream of events to be acted on, or for building denormalized state stores.

Using Apache Beam, this talk will present a few strategies for enriching unbounded sources with data from bounded and unbounded secondary sources. It will also discuss some of the tradeoffs between simplicity, consistency and complexity that we make.