trino icon indicating copy to clipboard operation
trino copied to clipboard

Suppress NoSuchFileException in LocalFileIterator

Open ebyhr opened this issue 1 year ago • 5 comments

Description

Files.walk method may throw an exception during the iteration. https://bugs.openjdk.org/browse/JDK-8039910 Fixes #20520

Release notes

(x) This is not user-visible or is docs only, and no release notes are required.

ebyhr avatar Feb 01 '24 02:02 ebyhr

Please get @electrum to review this before merging. The exact behavior of exceptions from the file system implementations is required to be consistent, and I'd like david to make sure this is following the expected behavior.

dain avatar Feb 01 '24 20:02 dain

So I believe that the first commit is correct and the second commit is not needed.

electrum avatar Feb 01 '24 21:02 electrum

@electrum Both commits are required to fix the concurrent issue. LocalFileIterator#next calls Files.size(path). It throws NoSuchFileException if the file is deleted concurrently.

ebyhr avatar Feb 02 '24 03:02 ebyhr

Thanks for explaining. We should change LocalFileIterator to ignore those exceptions by not returning the file entry. Listing files is not atomic, so this is an expected race condition that we need to handle.

electrum avatar Feb 02 '24 03:02 electrum

We should change LocalFileIterator to ignore those exceptions by not returning the file entry.

@ebyhr did you consider making use of a simplified version of java.nio.file.Files#walkFileTree(java.nio.file.Path, java.util.Set<java.nio.file.FileVisitOption>, int, java.nio.file.FileVisitor<? super java.nio.file.Path>) (what Files.walkFileTree uses in the background) and create the FileEntry instances on the fly in the LocalFileIterator?

findinpath avatar Feb 15 '24 13:02 findinpath