Metastore catalog

The metastore catalog is a concept that originated from the Hive project. The metastore stores an association between paths (initially on HDFS) and virtual tables.

A “table” in the metastore is made of:

Originally, a metastore catalog is an external service.

DSS features multiple integration points with the metastore catalog:

Multiple engines and features in DSS leverage the metastore (rather than the dataset definition) to perform computations:

DSS can leverage three kinds of metastores: