DirFileSystem not forwarding 'local_file'

[jdonnerstag]

My use case: I'm using the following snippet to create a virtual filesystem, and create an OpenFile handle. Important to me is that I have a common base class/interface (OpenFile) to pass on to my "business logic" because I've extended it already with types. fs.open() is returning different things depending on the fs, which by itself would be ok, but in combination with the requirement below, I prefer OpenFile over open().

fs = fsspec.implementations.dirfs.DirFileSystem("./sample_data/", fs=fsspec.filesystem("file"))
of = OpenFile(fs, my_file, mode="rb")

In my business logic I'm either loading the whole file in memory (bytes), or memory map it (large files). For this decision I need to know whether it is a local file (incl. cached locally) or not. Only local files can be mmapped. open_local() is testing the presence of a local_file attribute. Obviously a bit of a hack, and not yet part of a spec or contract. I tried to use the hack with the snippet above, but it doesn't work. DirFileSystem is not initialising a local_file attribute. So, right now I need to test for type DirFileSystem, if yes, then test for the local_file attribute, ...

I'm new to fsspec, so may be I'm missing a trick. I wish fsspec would have a clear API for local files (incl. locally cached files), e.g.:

   bool = of.has_local_file()   # True, if the file is local or have locally cached version
   fn = of.local_file()      # local file path. Raise exception if only remote

Both should return valid responses, even if the locally cached file does not yet exist. And they should not trigger a download.

[efiop]

This is my first time seeing local_file, but looks like dirfs could indeed simply pass it through similar to how we do that with sep in https://github.com/fsspec/filesystem_spec/blob/02c557c11d738eaf9d14ff5f350055dd2dd127ff/fsspec/implementations/dirfs.py#L48 @jdonnerstag Maybe you could try that out and if works contribute a patch?

[martindurant]

@jdonnerstag , do you think you might have time to contribute a fix?