Libraries like Ibis have been gaining traction recently, by unifying the way we work with data across multiple data platforms—from dataframe APIs to databases, from dev to prod. What if we could extend the abstraction to unify the way we work across batch and streaming contexts? In this presentation, we showcase our work adding streaming concepts and an Apache Flink backend to the open-source Ibis project.
Deepyaman is a software engineer at Voltron Data. Before their acquisition by Voltron Data, he was a Founding Machine Learning Engineer at Claypot AI, working on their real-time feature engineering platform. Prior to that, he led data engineering teams and asset development across a range of industries at QuantumBlack, AI by McKinsey.
Deepyaman is passionate about building and contributing to the broader open-source data ecosystem. Outside of his day job, he helps maintain Kedro, an open-source Python framework for building production-ready data science pipelines.