alluxio
alluxio copied to clipboard
Add folder check for load job
What changes are proposed in this pull request?
Add folder check for load job.
Why are the changes needed?
Adding 'partial-listing' when loading a file instead of a directory will cause the loading task to fail, as it results in duplicate blocks of the file being loaded.
./bin/alluxio fs load /test.txt --submit --partial-listing
./bin/alluxio fs load /test.txt --progress
Progress for loading path '/test.txt':
Settings: bandwidth: unlimited verify: false
Job State: FAILED (alluxio.exception.runtime.InternalRuntimeException: Job failed because it's not healthy.)
Files Processed: 5
Bytes Loaded: 2783.32MB
Throughput: 927.77MB/s
Block load failure rate: 5.02%
Files Failed: 1