Data Council Blog

Data Council Blog

Storing Cold Metadata, Snowflake Data Cloud, and More: Top 10 Links From Across the Web

Here's our January 2021 roundup of links from across the web that could be relevant to you:

1. Storing Cold Metadata with Alki (Dropbox)

Dropbox shared insights into Alki, the petabyte-scale metadata store it designed for infrequently accessed metadata (“cold data”). The post details how one-size-fits-all database Edgestore was reaching capacity limits, and why audit logs were a good candidate to be moved elsewhere than on costly SSDs. After considering off-the-shelf options, the team settled on building its own solution on top of AWS services: Alki; with DynamoDB as the hot store, and S3 as the cold store. Like HBase or Cassandra, Alki is based on log-structured merge-trees (LSM trees), but is better suited to handle hot-then-cold audit logs, as well as future use cases at Dropbox.

The Modern Data Stack, Metadata Architectures, and More: Top 10 Links From Across the Web

Here's our December 2020 roundup of links from across the web that could be relevant to you:

1. The Modern Data Stack (Fishtown Analytics)

This long-form post on the dbt blog is a must-read. Titled “The Modern Data Stack: Past, Present, and Future,” it answers the question that Tristan Handy has been asking himself for the past two years: “What happened to the massive innovation we saw from 2012-2016?” His carefully thought-out analysis covers the natural cycles of technological shifts, defines the phase we are in as a ‘deployment’ one, and points out high-impact opportunity areas for the next few years - which you might find particularly useful if you are considering launching a new product.

NLP Heroes, Pinot, Data Testing, and More: Top 10 Links From Across the Web

Here's our November 2020 roundup of good reads and podcast episodes that might be relevant for your career in data:

1. Heroes of NLP: Quoc Le (Deeplearning.ai)

NLP researcher Quoc Le was recently Andrew Ng’s guest as part of the ‘Heroes of NLP’ video series. Their discussion covered Le’s impressive journey, from growing up in Vietnam and developing his first basic chatbot in high school to becoming Google Brain’s first intern, and everything that followed. This includes the ‘Google Cat’ experiment, the Meena chatbot project, and work on Seq2Seq models. Check out the conversation here, and consider subscribing to the series to hear from other guests such as Chris Manning, Kathleen McKeown, and Oren Etzioni.

State of AI, Data Quality, and More: Top 10 Links From Across the Web

Here's our October 2020 roundup of good reads and podcast episodes that might be relevant to you as a data professional:

1. Multiplayer Editing: a Pragmatic Approach (Hex)

Data collaboration startup Hex published a great long read on its approach to live collaboration . Written by software engineer Mac Lockard, it takes a look at the respective pros and cons of Operational Transforms and Conflict-free Replicated Data Types (CRDTs), before explaining the solution that Hex adopted. Inspired by Figma's hybrid approach, it can also be described as "Atomic Operations (AO), as all edits to application state are broken down to their smallest atomic parts." "If the application you are building can rely on last-writer-wins semantics, Atomic Operations might provide a more pragmatic approach," the post concludes. This is a highly recommended read if you are pondering about a similar decision.  

Data Science, Data Analytics, Data Engineering and Artificial Intelligence: 11 Online Courses You Should Check Out

With COVID-19 forcing almost one billion people to shelter in place around the world, many people have turned to new activities, such as drawing, baking, gardening… or online learning. If that doesn't sound like you, don't feel guilty by any means – sometimes, surviving is enough! But if you want to get more knowledgeable about data science, data engineering and artificial intelligence, we are here for you.

This is why we came up with this list of courses that can help you prepare for a future job in the data field, upgrade your existing skills, or just satisfy your personal curiosity. From free entry-level courses to full-time bootcamps, here's our selection for you to check out:

Wanna be our Pen Pal?

Receive the latest news, tips and special events from our community directly to your inbox once in a while (we promise no spam)

Data Council Blog Signup