filesystem_spec
filesystem_spec copied to clipboard
Walk filesystem breaks if filesystem doesn't provide unique file names
If the filesystem provides files with same name at different directory then it breaks for example with HTTPFilesystem if the response is as follows:
"/folder1/file"
"/folder2/file"
Then it will only return the first file because the current implementation appears to assign them to the same key in a dictionary when walking and will only return the first one meaning I need to generate a unique file name for all of them.
Since HTTPFilesystem uses walk to implement glob this also means glob doesn't work