At Quid, our flagship product is a SaaS research platform which allows users to gain full-picture insights about written content like news. Over the past year, we built an internal framework so that data scientists and data engineers can encapsulate tailored product use cases (e.g., "understand how a company and its competitors are being covered in the media" or "find key opinion leaders in the news related to some topic") into single apps. This framework has allowed for the modularization of data science capabilities and has dramatically lowered the effort needed to see data science realized in application form. We use Netflix's open-source orchestration system, Conductor, to isolate infrastructure concerns and allow data science teams to safely own products from prototype to production.
In this talk I'll cover:
* Big-picture transferable lessons that we’ve learned the hard way, that you can apply yourself to solve your team’s data science workflow challenges
* How the structure of our data science and data engineering team has evolved to utilize this framework
* How we leverage data from our flagship product to discover what apps to build next
Austen is a senior data scientist at Quid and a data science advisor at Halo and at Patreon. Previously, Austen received a PhD in Statistics from Stanford, was the first employee at the consumer fintech company Earnest, and then was an early employee at the enterprise biotech sales company Halo. His work has focused on developing data products from complex data structures which are highly understandable by non-data-savvy users