alluxio icon indicating copy to clipboard operation
alluxio copied to clipboard

Add unbuffer support

Open qian0817 opened this issue 2 years ago • 2 comments

Is your feature request related to a problem? Please describe. Implement the unbuffer interface for HdfsFileInputStream. This is very helpful for impala's file handle cache.

Describe the solution you'd like' Implement the unbuffer interface for HdfsFileInputStream.

Describe alternatives you've considered

Urgency

Additional context

qian0817 avatar Aug 05 '22 02:08 qian0817

Hey @qian0817 , do you mind giving some performance analysis of the change, otherwise, it's hard to understand the impact.

HelloHorizon avatar Aug 10 '22 17:08 HelloHorizon

Impala Provide the ability to cache the file handle, instead of closing it immediately after opening and finishing reading. it will use the unbuffer method to release the file handle buffer. This can improve performance by reducing the number of openFile RPC calls when open the same file frequently. If the CanUnbuffer interface is not implemented, then Impala will not cache the file handle.

qian0817 avatar Aug 11 '22 02:08 qian0817

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in two weeks if no further activity occurs. Thank you for your contributions.

github-actions[bot] avatar Feb 01 '23 15:02 github-actions[bot]