A/B testing is a core component of the product development lifecycle, applying the scientific method to building the right things for our customers. It is a deceptively simple concept - randomize users into one of a few competing experiences, measure the impact to key metrics, compare the experiences and make a decision on what to ship. Drawing from years of experiences running A/B tests in the wild, I will delve into the practical considerations behind designing and running effective tests (hint: it's not found in a statistics textbook!). Practitioners will walk away with advice on avoiding common pitfalls and ensuring that their A/B tests drive real impact.