presto icon indicating copy to clipboard operation
presto copied to clipboard

Fix an issue while accessing s3 when RaptorX is enabled

Open imjalpreet opened this issue 2 years ago • 2 comments

This is needed because RaptorX/Alluxio only works with URI schemes alluxio:// and ws://. This scheme is only used as a key for collecting stats.

Fixes #15951

== NO RELEASE NOTE ==

imjalpreet avatar Jul 15 '22 10:07 imjalpreet

One minor comment, it is ok, if you want to defer it.

arunthirupathi avatar Jul 19 '22 18:07 arunthirupathi

Shouldn't the right fix be removing the check here https://github.com/Alluxio/alluxio/blob/master/core/client/hdfs/src/main/java/alluxio/hadoop/LocalCacheFileSystem.java#L91

The supported scheme is not used beyond that.

Upgrading the dependency after the fix is released.

CC: @rongrong who is a contributor to the Alluxio Caching Filesystem code.

arunthirupathi avatar Jul 20 '22 01:07 arunthirupathi

@rongrong Hi! Could you please comment the PR?

dnskr avatar Aug 28 '22 18:08 dnskr

@arunthirupathi @jainxrohit Can you please provide an update on this? Were you able to test it internally?

@rongrong Can you please provide your inputs on this comment https://github.com/prestodb/presto/pull/18040#issuecomment-1189719660

imjalpreet avatar Sep 12 '22 09:09 imjalpreet

Based on our discussion with Bin Fan (Alluxio), I will be closing this PR as the Alluxio team is in discussions(looking positive) to include S3 and HDFS(to begin with) in the supported filesystems for LocalCacheFileSystem (Alluxio) and it is estimated to be released in a month or two barring any setbacks. Once this change is released, we won't need the changes in this PR. Feel free to reach out in case of any questions.

imjalpreet avatar Nov 03 '22 14:11 imjalpreet