The increasing prevalence of A|B Testing has revealed a powerful, but seldom discussed insight: Our intuitions are soberingly bad! Yet, most of us are either unaware or chose not to believe this. Time and time again, randomized controlled trials reveal that our predictions on the outcomes of business decisions are deeply fallible: Experiments show that our optimsm in new features is often misguided; what we thought would help actually hurts; and what we believed to be the next big thing fails completely. Our intuitions can even lead to worse outcomes than what we would expect from random chance alone.
This is not a talk about AB Testing, but rather what empiricism reveals about our intuitions. We describe some types of errors domain experts commit. We discuss how these systematic errors arise from heuristic reasoning processes that served us well on an evolutionary time scale, but now impair our judgments. Using interactive audience demonstrations we illustrate how cognitive heuristics can compromise decisions and their evaluation.
While we can’t change how our brains are wired, knowledge of how they work can give us an advantage. There are several mechanisms to mitigate these human limitations and improve our decision-making. We believe that businesses which make fewer bad decisions will have a distinct competitive advantage.
Daragh leads a team of Data Scientists that are focused on optimizing the portfolio of products stocked in Stitch Fix's inventory. They use algorithms and the scientific method to leverage Stitch Fix’s large and unique dataset about client preferences, purchasing behavior, and merchandise attributes. Prior to Stitch Fix, Daragh spent a decade in academia, where he developed neural networks of human language acquisition and tested their predictions with behavioral and neuroimaging experiments. He holds a Ph.D.Applied Experimental Psychology.
Eric is the Chief Algorithms Officer at Stitch Fix, leading a team of 100+ Data Scientists. He is responsible for the multitude of algorithms that are pervasive to nearly every function of the company: merchandise, inventory, marketing, forecasting & demand, operations, and the styling recommender system. Prior to joining Stitch Fix, he was the Vice President of Data Science & Engineering at Netflix. Eric holds a B.A. in Economics, an M.S. in Information Systems, M.S. in Management Science & Engineering.