Your queries won't run fast if your data is not organized right.
Apache Calcite optimizes queries, but can we evolve it so that it can
optimize data? We had to solve several challenges. Users are too busy
to tell us the structure of their database, and the query load changes
daily, so Calcite has to learn and adapt. We talk about new algorithms
we developed for gathering statistics on massive database, and how we
infer and evolve the data model based on the queries, suggesting
materialized views that will make your queries run faster without you
Julian Hyde is an expert in query optimization, database internals, in-memory analytics, and streaming. He is the founder of Apache Calcite, an open-source query planning framework that powers many database and streaming SQL engines, including Apache Beam, Flink and Hive. He was the original developer of the Mondrian OLAP engine, and was formerly Chief Architect at SQLstream. He is an architect at Looker.