Experimentation is a critical part of the modern product development lifecycle. In order to implement experimentation effectively, data teams must invest in robust, scalable assignment systems, metrics computation, and complex statistical analysis frameworks. Most companies get started with decentralized analysis, in Jupyter or mode notebooks. Others centralize piecemeal with a looker dashboard and homegrown feature flagging, but leave crucial workflows to be done ad hoc or not at all. This talk will review the core elements of the modern experimentation stack, which components must be closely vetted, and examples of how bad experimentation can be more damaging to an organization than no experimentation at all. Listeners will leave with an understanding of how to spot a faulty experimentation stack with a review of alternative solutions.
Chad Sanderson is the current Head of Product for Data Platform at Convoy and has worked on Big Data and AI for Microsoft, Sephora, Subway, and Oracle. He is a passionate believer in the power of the scientific process and the value of measurement.