Technical Talks

View All

How to Optimize Apache Spark Cloud Clusters for Cost and Runtime Goals with Sync Computing

Jeffrey Chou Jeffrey Chou | Co-Founder & CEO | Sync Computing
Kartik Nagappa Kartik Nagappa | Staff Product Manager | Sync Computing

Optimizing Apache Spark clusters on EMR and Databricks to hit cost goals is a time-consuming task, but is an increasingly important task in today’s economy. Specifically for production workloads on AWS, optimizing instance types, spark configurations, and cluster sizes is an incredibly complex process. The Sync Autotuner for Apache Spark helps to solve this problem by predicting workload performance on alternative cloud infrastructure and configurations. In this talk developers will learn how to increase productivity, reduce cloud costs, and improve performance of the production Apache Spark workloads. In the second half we’ll walk users through a live hands-on demo of the Autotuner product.

Jeffrey Chou
Jeffrey Chou
Co-Founder & CEO | Sync Computing

Jeff received his PhD in EECS from UC Berkeley as an NDSEG fellow, was a Batelle Post-Doctoral Scholar at MIT, and is an Entrepreneurial Research Fellow at Activate. Prior to Sync, he was a technical staff member at MIT Lincoln Laboratory.

Kartik Nagappa
Kartik Nagappa
Staff Product Manager | Sync Computing

Kartik vaguely recalls writing a thesis on polyphase recursive all-pass filters for his masters in digital signal processing. Since then, he has switched gears and worked at several companies like Microsoft, Netflix and other startups specializing in Product Management.