presto
presto copied to clipboard
Procedure to invalidate directory list cache
Test plan - (Please fill in how you tested your changes)
Please make sure your submission complies with our Development, Formatting, and Commit Message guidelines. Don't forget to follow our attribution guidelines for any code copied from other projects.
Fill in the release notes towards the bottom of the PR description. See Release Notes Guidelines for details.
== RELEASE NOTES ==
Hive Changes
* Introduce system procedure to invalidate directory list cache in Hive Catalog
@agrawalreetika Instead of single directory, we may want to clear the whole cache too. May be make the directoryPath optional, and if its not provided, then clear all cache. Also, please change the procedure name to something more intuitive.
If there are multiple hive catalogs, how can we invalidate the cache from one particular catalog?
system.invalidate_file_list_cache(Optional<String> directoryPath)
@agrawalreetika Instead of single directory, we may want to clear the whole cache too. May be make the directoryPath optional, and if its not provided, then clear all cache. Also, please change the procedure name to something more intuitive.
If there are multiple hive catalogs, how can we invalidate the cache from one particular catalog?
system.invalidate_file_list_cache(Optional<String> directoryPath)
Hi @nmahadevuni, Thanks for your comment. I have added the directory path as optional to allow full directory list cache invalidation. Just like any other procedure in Presto we will have to provide catalog name while making procedure call on specific catalog. (ref - https://prestodb.io/docs/current/connector/hive.html#procedures)
@tdcmeehan Could you please help me review this?
Codenotify: Notifying subscribers in CODENOTIFY files for diff 908327e1e99bdf4728aa6cbf9f7eb5d5c94d7737...004ba4bf4ceea9f0dc9b39293cb281fd958af13f.
Notify | File(s) |
---|---|
@steveburnett | presto-docs/src/main/sphinx/connector/hive.rst |
Failing on one test (test / test (:presto-hive) (pull_request), and no option to re-run failed tests.
@tdcmeehan I have been facing this issue in product-tests-basic-environment
here but not finding the details around what's causing this.
Are there any more logs that get captured as part of workflow run?
@agrawalreetika why do we need new product tests for this feature? Can't it run in an existing suite?
@agrawalreetika why do we need new product tests for this feature? Can't it run in an existing suite?
@tdcmeehan I added tests to the existing suite, but looks like product-tests-basic-environment
is having same issue here
@tdcmeehan & @steveburnett all the tests are green, this is ready for final review.