presto icon indicating copy to clipboard operation
presto copied to clipboard

Manifest file caching support for Iceberg Hive Catalog

Open agrawalreetika opened this issue 1 year ago • 2 comments

Description

Add Manifest file caching support for Iceberg Hive Catalog

Motivation and Context

Starting from Iceberg version 1.1.0, Apache Iceberg provides a mechanism to cache the contents of Iceberg manifest files in memory. This manifest caching feature helps to reduce repeated reads of small Iceberg manifest files from remote storage. Reference: https://github.com/apache/iceberg/pull/4518

Impact

No Impact. Default behaviour, Disabled. Feature can be enabled via catalog configuration changes.

Test Plan

Contributor checklist

  • [ ] Please make sure your submission complies with our development, formatting, commit message, and attribution guidelines.
  • [ ] PR description addresses the issue accurately and concisely. If the change is non-trivial, a GitHub Issue is referenced.
  • [ ] Documented new properties (with its default value), SQL syntax, functions, or other functionality.
  • [ ] If release notes are required, they follow the release notes guidelines.
  • [ ] Adequate tests were added if applicable.
  • [ ] CI passed.

Release Notes

Iceberg Changes
* Add Manifest file caching support for Iceberg Hive Catalog

agrawalreetika avatar Feb 05 '24 13:02 agrawalreetika

CLA Signed

The committers listed above are authorized under a signed CLA.

  • :white_check_mark: login: agrawalreetika / name: Reetika (c7f149b297b8b73247672e629eba1dde1b29dffe)

Codenotify: Notifying subscribers in CODENOTIFY files for diff 6998a7362764a73186a13a713a651ec71314c5f7...c7f149b297b8b73247672e629eba1dde1b29dffe.

Notify File(s)
@steveburnett presto-docs/src/main/sphinx/connector/iceberg.rst

github-actions[bot] avatar Feb 06 '24 04:02 github-actions[bot]

Please merge when you get a clean test run

tdcmeehan avatar Mar 06 '24 00:03 tdcmeehan

Blocked on https://github.com/apache/iceberg/issues/9991

tdcmeehan avatar Mar 18 '24 14:03 tdcmeehan