Developing source of truth data and metrics has been a perpetual quest at Airbnb. As complexity and the size of the company grows, the number of key metrics and dimensional cuts teams track grows, as well. Tribal knowledge of important data does not scale to large teams, and all teams may not have the data engineering skills to build and maintain pipelines.
To scale metric definition and discovery, we built a metrics framework to be the source of truth for metric definitions. This framework calculates metrics daily, feeds those metrics to reporting, experimentation, and anomaly detection applications, makes the searchable in Airbnb's Dataportal, and abstracts data lifecycle management issues such as schema evolution and backfilling. This solves the challenge of inconsistent metrics definitions, improving efficiency of Data Scientists defining metrics and freeing them from pipeline management. We have seen rapid adoption of this framework, ultimately driving business value through increasing the trustworthiness and availability of metrics.
