presto icon indicating copy to clipboard operation
presto copied to clipboard

Add support for Filelist caching for symlink tables

Open agrawalreetika opened this issue 1 year ago • 4 comments

Test plan - (Please fill in how you tested your changes)

Using the hive catalog access the symlink tables by adding following properties:

hive.file-status-cache-expire-time=24h
hive.file-status-cache-size=100000000
hive.file-status-cache-tables=*

Metrics Queries -

presto> SELECT * FROM jmx.current."com.facebook.presto.hive:name=<catalog>,type=cachingdirectorylister";
presto> SELECT "listlocatedstatuscalls.totalcount", "listobjectscalls.totalcount", "liststatuscalls.totalcount" from jmx.current."com.facebook.presto.hive.s3:name=<catalog>,type=prestos3filesystem";

== RELEASE NOTES ==

Hive Changes
* Add support for Filelist caching for symlink tables

agrawalreetika avatar Mar 07 '23 07:03 agrawalreetika

I'm sorry, I missed this earlier. @zhenxiao, I have updated the commit with tests. Please review.

agrawalreetika avatar Jun 19 '23 18:06 agrawalreetika

Hi @zhenxiao, Could you please take a pass on this?

agrawalreetika avatar Jun 30 '23 05:06 agrawalreetika

@zhenxiao Looks like after rebasing, the Jmx values in the test are not coming up after caching. But earlier these JMX metrics were populating fine. And this test runs fine from my Local IDE.

agrawalreetika avatar Jul 13 '23 06:07 agrawalreetika

yep, let's try fix the testcase. Then we could merge :)

zhenxiao avatar Jul 13 '23 19:07 zhenxiao