How to Ensure Your Model Does Not Drift? From Human-In-The-Loop Concept to Building Fully Adaptive Ml Models Using Crowdsourcing

Fedor Zhdanov, Head of AI | Toloka

All ML models will degrade over time due to drifts in the statistical properties of predicted variables or predictors that the model uses. It can happen within several months or even years since the model is trained, or it can happen within several hours after it was deployed to production.

In this talk, Fedor will explain how to combat the concept of drift in ML with crowdsourcing. He will show you how to build complex drift-monitoring systems and human-in-the-loop ML models that can be fully automated. He will also tell you how this has led him and his team to start building so-called “adaptive ML models”.

You will learn what they are, how to build and maintain them.


Fedor Zhdanov

Head of AI | Toloka

Fedor is the Head of AI at Toloka. Before that, he held Principal Applied Scientist position at AWS, and previously worked for Microsoft and Amazon. Fedor has been creating products with R&D in Machine Learning for the last 18+ years. For the last 6 years, he has been focusing on connecting ML and humans in human-in-the-loop processes. His ventures are focused on building responsible state of the art AI-first business solutions with human oversight. Fedor holds a Ph.D. in Computer Science from Royal Holloway, University of London.

Fedor Zhdanov