Streaming applications can be designed to balance or favor one or more of latency, throughput, memory consumption, or CPU load. In order to scale growing real-time applications well, properties like replayability, at-least-once and exactly-once processing, and out-of-order processing drive decisions that need to be made inside the streaming application and by data producers and consumers.
This presentation discusses some useful design patterns for streaming applications that help deliver great value and an exceptional digital personalization experience to our customers, with personalized responses for Capital One's Eno chatbot as an example.
Andreas Markmann is Manager of Data Engineering at Capital One and tech lead for the Potomac Clickstream project, where he works with engineers and partner teams to democratize strongly scaling data efficiently for the benefit of customer experience. Before joining Capital One, he created efficient parallel classical and quantum dynamics algorithms for the simulation of laser-molecule and nuclear fusion reactions. He has a PhD in Theoretical Physics from University College London and an MSc in Pure Mathematics from Queen Mary and Westfield College London. He financed his undergraduate degree improving his English as a sightseeing tour guide in Berlin, Germany and volunteered as a rock climbing, swimming, and programming coach for underserved communities.