alluxio
alluxio copied to clipboard
Add unbuffer support
Is your feature request related to a problem? Please describe.
Implement the unbuffer interface for HdfsFileInputStream
. This is very helpful for impala's file handle cache.
Describe the solution you'd like'
Implement the unbuffer interface for HdfsFileInputStream
.
Describe alternatives you've considered
Urgency
Additional context
Hey @qian0817 , do you mind giving some performance analysis of the change, otherwise, it's hard to understand the impact.
Impala Provide the ability to cache the file handle, instead of closing it immediately after opening and finishing reading. it will use the unbuffer
method to release the file handle buffer. This can improve performance by reducing the number of openFile RPC calls when open the same file frequently. If the CanUnbuffer
interface is not implemented, then Impala will not cache the file handle.
This issue has been automatically marked as stale because it has not had recent activity. It will be closed in two weeks if no further activity occurs. Thank you for your contributions.