Technical Talks

View All

Building an Ecosystem for Open Foundation Models, Together

Ce Zhang Ce Zhang | CTO | Together

In this talk, I hope to share insights and experiences from our collaboration with the community to enhance open source foundation model ecosystems. A primary opportunity (and challenge) lies in balancing and jointly optimizing data quality, model architecture, and infrastructure. This includes managing the vast scale and cost of GPU clusters, optimizing their use, and reasoning about data quality in a principled manner to enhance model quality. To this end, we have focused our efforts on several technical problems, such as developing the RedPajama dataset, which tries to provide a modular perspective on data quality; communication optimization algorithms to accelerate learning across disaggregated infrastructures; and optimized inference infrastructure through the deep co-design of systems and model architecture. In this talk, I will describe our learnings from some of these projects and hope to receive feedback from everyone on how we can collectively advance the open source foundation model ecosystem.

Ce Zhang
Ce Zhang
CTO | Together

Ce is CTO at Together and an Associate Professor in Computer Science at ETH Zurich. The mission of his research is to make machine learning techniques widely accessible-‚Äč‚Äč‚Äč-‚Äč-‚Äčwhile being cost-‚Äč‚Äčefficient and trustworthy-‚Äč‚Äč‚Äč-‚Äč-to everyone who wants to use them to make our world a better place. He believes in a system approach to enabling this goal, and his current research focuses on building next-‚Äč‚Äčgeneration machine learning platforms and systems that are data-‚Äč‚Äčcentric, human-‚Äč‚Äčcentric, and declaratively scalable. Before joining ETH, Ce finished his PhD at the University of Wisconsin-‚Äč‚Äč‚ÄčMadison and spent another year as a postdoctoral researcher at Stanford, both advised by Christopher R√©. His work has received recognitions such as the SIGMOD Best Paper Award, SIGMOD Research Highlight Award, Google Focused Research Award, an ERC Starting Grant, and has been featured and reported by Science, Nature, the Communications of the ACM, and a various media outlets such as Atlantic, WIRED, Quanta Magazine, etc.