filesystem_spec icon indicating copy to clipboard operation
filesystem_spec copied to clipboard

Walk filesystem breaks if filesystem doesn't provide unique file names

Open jcampbell05 opened this issue 1 year ago • 4 comments

If the filesystem provides files with same name at different directory then it breaks for example with HTTPFilesystem if the response is as follows:

"/folder1/file"
"/folder2/file"

Then it will only return the first file because the current implementation appears to assign them to the same key in a dictionary when walking and will only return the first one meaning I need to generate a unique file name for all of them.

Since HTTPFilesystem uses walk to implement glob this also means glob doesn't work

jcampbell05 avatar Jul 11 '24 13:07 jcampbell05