Growing startups all reach a point at which they need to be more data driven to provide incremental value for end users. To do so, startups leverage machine learning algorithms which require them to transform source data into insights. This transition often requires a re-architecture of production systems into a big data architecture in order to consolidate the data and ensure it is immutable. Working in parallel, leveraging new startup technologies and python’s open source community, we have built a framework that can incrementally provide actionable insights without large code refactoring. This talk will walk through our journey in making data a first class citizen at WeWork.
Tarush Aggarwal works in the Data Engineering team at WeWork which works on creating the next gen big data load, compute, store, interact, and BI platforms. He is very interested in analysis and visualization of Log Data. He received his bachelor’s degree in Electrical and Computer Engineering from Carnegie Mellon University in 2011. He was heavily involved in Research work at the Parallel Data lab working on frameworks based around Indexing and Retrieval of Celestial Objects.