After two years, three rounds of funding, and hundreds of new employees — Whatnot’s modern data stack has come from not existing to processing tens of millions of events across hundreds of different event types each day. How does our small (but mighty!) team keep up? This talk explores data contracts — it covers how we use an Interface Definition Language (Protobuf) to serve as the source of truth for event definitions, govern event construction in production, automatically generate DBT models in the data warehouse, and more.
Zack Klein is a software engineer at Whatnot, where he thoroughly enjoys building data products and narrowly avoiding breaking production each day. Previously, he worked on big data platforms at Blackstone and HBO. When he's not coding or writing, you can find him playing his telecaster, watching the San Francisco Giants, or listening to Dawes.