At Materialize, Inc. we are building a high-throughput, low-latency SQL view maintenance engine. You write SQL queries against continually evolving relations, we give you back the answers fast. You ask the queries again and you get updated answers in milliseconds.
This system design departs fundamentally from both Spark-like and relational database systems, and is based instead on timely dataflow and differential dataflow. In this talk, we will go through the architectural highlights distinguish Materialize from prior systems, call out how they enable interactive queries over continually evolving data, and demonstrate the stack used for real-time data warehousing.
Frank McSherry is Chief Scientist at Materialize, where he (and others) convert SQL into scale-out, streaming, and interactive dataflows. Before this, he developed the timely and differential dataflow Rust libraries (with colleagues at ETHZ), and led the Naiad research project and co-invented differential privacy while at MSR Silicon Valley. He has a PhD in computer science from the University of Washington.