As a field, we often hear about success stories. This is true in research, where a publishing incentive can pressure authors to focus on consistently exceeding state of the art results. It is also true in industry, where companies attempt to attract engineering talent by describing how impressive their production ML systems are.
However, every practitioner here knows that in engineering and in ML, the road to success is paved with failures. The field of ML in production is new, and so has a lack of cautionary tales of things that can go wrong with models. This talk will try to help correct that.
We will discuss challenges such as performance mismatch between offline training and online inference, feature generation and data leakage, and adequate roadmap planning for ML.
Emmanuel Ameisen currently is an ML Engineer at Stripe. Recently, Emmanuel led Insight Data Science's AI program where he oversaw more than a hundred machine learning projects. He previously implemented and deployed predictive analytics and machine learning solutions for Local Motion and Zipcar. Emmanuel holds graduate degrees in artificial intelligence, computer engineering, and management from three of France’s top schools.