Creating our own Kubernetes and Docker to run our data infrastructure

Erik Bernhardsson, Founder | Modal Labs

When Erik started working on an infrastructure solution for data teams, he quickly realized that Docker and Kubernetes were too slow and not right for the use case. Almost everyone said it was a bad idea to build their own system, but they ended up building it anyway. In this talk, Erik will share how Modal starts 1000s of large containers in seconds, and what they had to do under the surface to build this. This includes a custom file system written in Rust, their own container runtime, and their own container image builder. This talk will give you an idea of how containers work along with some of the low-level Linux details underneath. But along the way we'll also talk about many infrastructure tools hold data teams back, and why they deserve faster and better tools.

ABOUT THE SPEAKER

Erik Bernhardsson

Founder | Modal Labs

Erik is the founder and CEO of Modal, which is an infrastructure provider for data teams. Before Modal, Erik was the CTO at Better for six years, and previously spent seven years at Spotify, building the music recommendation system and running data teams.

Erik Bernhardsson