Huge progress has been made in commodifying both predictive modeling and A/B testing. Less progress has been made in productizing so-called structural models---models based on sufficiently rigorous microfoundations to be able to make informed judgements about the behavior of a system away from the training cases.
A common case of where structural models outperform standard predictive models is in predicting sales for a new product in a competitive market. We might train a model to predict sales based on product characteristics and market conditions. Yet the training data won't typically provide cases of the response that competitors will make in response to the entry of your product.
We must jointly predict the strategic behavior of competitors as well as prospective sales given their strategic response. This necessitates specifying the structural problem that all agents are solving. Jim will provide an example of such model, and talk through the steps required to take these approaches to industry-scale data in a productized way.
James Savage is an applied modeler and Data Science Lead at frontier markets lender Lendable in New York City. Previously he was at the Grattan Institute, La Trobe University, and the Australian Treasury. He is currently writing a book on Bayesian Econometrics in Stan.