Technical Talks

Julien Le Dem

Principal Engineer | Datadog

Ten years of building open source standards: From Parquet to Arrow to OpenLineage

Data Eng & Infrastructure

Over the last decade I have been lucky enough to contribute a few successful open source projects to the data ecosystem. In this talk I’ll share the story of how these projects came to be and what made their success possible. I’ll describe the ideation process and early growth of the Apache Parquet columnar format and show how this led to the creation of its in-memory alter-ego Apache Arrow. I’ll end with showing how this experience enabled the success of OpenLineage, an LFAI & Data project that brings observability to the data ecosystem. Along the way, I’ll talk about the key elements that catalyzed their growth, from project focus to governance to community.

Download Slides

Principal Engineer

Julien Le Dem

Datadog

Julien Le Dem is a Principal Engineer at Datadog, serves as an officer of the ASF and is a member of the LFAI&Data Technical Advisory Council. He co-created the Parquet, Arrow and OpenLineage open source projects and is involved in several others. His career leadership began in Data Platforms at Yahoo! - where he received his Hadoop initiation - then continued at Twitter, Dremio and WeWork. He then co-founded Datakin (acquired by Astronomer) to solve Data Observability. His French accent makes his talks particularly attractive.