filesystem_spec icon indicating copy to clipboard operation
filesystem_spec copied to clipboard

AbstractFileSystem.exists() catches every exception

Open tfelbr opened this issue 1 year ago • 1 comments

Hello,

I would like to kindly request a change in the implementation of AbstractFileSystem.exists(), as it currently catches every exception that might get thrown without any notice. An in my opinion better approach would be something like this:

def exists(self, path, **kwargs):
    try:
        self.info(path, **kwargs)
        return True
    except FileNotFoundError:
        return False
    except Exception as e:
        logging.warning(e)
        return False

This would preserve the current compatibility while still showing thrown exceptions as a warning.

tfelbr avatar Sep 28 '23 11:09 tfelbr

Warnings are only really useful if they imply a user can meaningfully act on them - which would not be the case here. Perhaps a logging call would be apropriate.

Note that it is not, in general, possible to know whether a file cannot be reached right now or if indeed it doesn't exist, especially with backends that don't have formal listing capabilities (e.g., HTTP). That means that we must interpret any number of IO and parsing-related exceptions as non-existence, in the sense that the path points to content we cannot read.

martindurant avatar Sep 28 '23 13:09 martindurant