alluxio
alluxio copied to clipboard
Alluxio Fuse should improve the sequential write handling with Libfuse 3
Is your feature request related to a problem? Please describe. It is possible the workload writes sequentially a file with a large buffer, eg 1M, in Fuse side, the offset is not handled in order, it could be a slightly out of order, for example, the second 128k of 1M is firstly handled by Fuse thread, then the first 128k of 1M is shown. So far Alluxio Fuse will just error out.
Describe the solution you'd like Alluxio Fuse can simply buffer the data received out of order, and write them if all data ahead in logic offset is received.
Describe alternatives you've considered It is possible to check if the fuse kernel buffer can be increased to cater the same size of write https://github.com/libfuse/libfuse/blob/master/ChangeLog.rst#libfuse-360-2019-06-13 But that depends on the linux kernel
Urgency This is important
Additional context Fuse debug log which is indicating out of order write request write[4] 131072 bytes to 47054848 flags: 0xc002 write[4] 131072 bytes to 46399488 flags: 0xc002 write[4] 131072 bytes to 47054848 write[4] 131072 bytes to 46399488
@yyongycy did you see this behavior again? and is it reproducible? are we able to close this issue?
This is tested with libfuse3, maybe keep it open and add libfuse3 label there.
I reproduce this bug.