delta icon indicating copy to clipboard operation
delta copied to clipboard

Set time travel in Hive Metastore catalog for Delta Lake tables.

Open alxvsk opened this issue 1 year ago • 0 comments

Ability to set time travel option in Hive Metastore catalog for HMS-managed Delta Lake tables. https://github.com/delta-io/delta/issues/2163

  • [X ] Spark
  • [ ] Standalone
  • [ ] Flink
  • [ ] Kernel
  • [ ] Other (fill in here)

Description

Need to provide time travel options from the metadata catalog to deliver users a historical view of the table if the time travel option is not explicitly specified in Spark. For example, Hive Metastore / Catalog will provide either versionAsOf or timestampAsOf SerDe parameters (similar to path, which is already stored in SerDe parameter in table’s Storage Descriptor to specify location of Delta table) to achieve this goal. Time travel options from the Hive Metastore catalog will work the same as if it was set from Delta format options, only one versionAsOf or timestampAsOf is allowed.

How was this patch tested?

This patch was tested with Hive metastore and covered with Delta test cases.

Does this PR introduce any user-facing changes?

No changes, unless table serde parameters are specified via Catalog / Hive metastore

alxvsk avatar Dec 21 '23 17:12 alxvsk