cloudpathlib
cloudpathlib copied to clipboard
LocalS3Path doesn’t support glob on non-existing directories
The implementations of S3Path and LocalS3Path diverge in the following way which disqualifies using LocalS3Path as a testing tool.
from cloudpathlib.s3 import S3Path
path = S3Path("s3://this/path/does/not/exist")
path.mkdir(parents=True) # does nothing (because S3 doesn't have directories)
print(list(path.glob("*"))) # returns []
from cloudpathlib.local import LocalS3Path
path = LocalS3Path("s3://this/path/does/not/exist")
path.mkdir(parents=True) # again, does nothing
print(list(path.glob("*"))) # raises exception
To fix this in my own codebase, I added a check to LocalS3Path.glob to return an empty generator when the path is not a directory. However, I’m unsure if this fixes the issue in all cases and there might be other issues caused by directories not existing in S3 but locally.
Verified this bug.
I'd be happy with a fix that overrides glob in the LocalPath implementation to try globbing and return an empty generator on FileNotFoundError.