FSFile support for avhrr l1b gaclac

[carloshorn]

This PR enables the use of FSPath objects for the avhrr_l1b_gaclac reader.

• [ ] Closes #1299 (for my use case)
• [x] Tests added
• [x] Passes flake8 satpy
• [ ] Fully documented
• [ ] Add your name to AUTHORS.md if not there already

Note:

Beside fsspec FileSystem paths, this PR allows any sort of PathLike objects as filenames. This allows passing file objects by defining a PathLike class as follows:

@total_ordering
class FileObjectPath(os.PathLike):
    def __init__(self, fileobj, filepath=None):
        self._fileobj = fileobj
        self._filepath = filepath or fileobj.name
    
    def __str__(self):
        return self._filepath
    
    def __fspath__(self):
        return self._filepath
    
    def __lt__(self, other):
        return os.fspath(self) < os.fspath(other)
    
    def open(self):
        return self._fileobj

[codecov[bot]]

Codecov Report

Merging #1470 (8b48215) into master (c0e21e6) will increase coverage by 0.00%. The diff coverage is 100.00%.

@@           Coverage Diff           @@
##           master    #1470   +/-   ##
=======================================
  Coverage   90.58%   90.58%           
=======================================
  Files         239      239           
  Lines       34313    34339   +26     
=======================================
+ Hits        31081    31107   +26     
  Misses       3232     3232           

Flag	Coverage Δ
behaviourtests	4.53% <0.00%> (-0.01%)	:arrow_down:
unittests	91.06% <100.00%> (+<0.01%)	:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
satpy/readers/avhrr_l1b_gaclac.py	95.73% <100.00%> (+0.07%)	:arrow_up:
satpy/tests/reader_tests/test_avhrr_l1b_gaclac.py	99.66% <100.00%> (+0.02%)	:arrow_up:

---

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update c0e21e6...8b48215. Read the comment docs.

[mraspaud]

Looks good so far! The tests are missing obviously, and I was thinking that providing a code snippet as an example on how to pass a zipped file to the reader would help?

[carloshorn]

Example: Reading from zip Archive

Imagine having a zip archive with the following structure

Archive:  data/archive.zip
Zip file size: 118719147 bytes, number of entries: 3
-rw-r-----  3.0 unx 63558144 bx defN 16-Jun-09 15:30 NSS.GHRR.NN.D16160.S0504.E0659.B5694748.WI
-rw-r-----  3.0 unx 63558144 bx defN 16-Jun-09 15:30 NSS.GHRR.NN.D16160.S0653.E0848.B5694849.WI
-rw-r-----  3.0 unx 63558144 bx defN 16-Jun-09 15:30 NSS.GHRR.NN.D16160.S0842.E1037.B5694950.WI
3 files, 190674432 bytes uncompressed, 118718489 bytes compressed:  37.7%

and you want to read the file NSS.GHRR.NN.D16160.S0504.E0659.B5694748.WI without extracting it from the archive. In this situation, this PR allows to use the following implementation

import satpy
from satpy.readers import FSFile
from fsspec.implementations.zip import ZipFileSystem

archive = ZipFileSystem('data/archive.zip')
filepath = 'NSS.GHRR.NN.D16160.S0504.E0659.B5694748.WI'

bands = ['1', '2', '3a', '3b', '4', '5']
aux_data = ['latitude', 'longitude', 'qual_flags',
            'sensor_zenith_angle', 'solar_zenith_angle',
            'solar_azimuth_angle', 'sensor_azimuth_angle',
            'sun_sensor_azimuth_difference_angle']
filename = FSFile(filepath, fs=archive)
reader = 'avhrr_l1b_gaclac'
reader_kwargs = {'tle_dir': './TLE/',
                 'tle_name': 'TLE_%(satname)s.txt',
                 'tle_thresh': 7}
scene = satpy.Scene(filenames=[filename], reader=reader,
                    reader_kwargs=reader_kwargs)
scene.load(bands)
scene.load(aux_data)

[carloshorn]

I could imagine such an example as test, where the FileSystem and the pygac reader instance would be mocks. This should hopefully only test the feature and not the implementation. What do you think @mraspaud?

[carloshorn]

The implementation of reading PathLike objects with an open method has been moved to https://github.com/pytroll/pygac/pull/92. Once it is merged on pygac, this PR becomes obsolete.

[mraspaud]

Ok with closing this then?

[carloshorn]

Ok with closing this then?

Would be okay from my side, bearing in mind that we wanted to put the example of reading from a zip archive into the docs.

[mraspaud]

Ok, yes, so we can use this PR for the documentation, is that what you are hinting at?

[carloshorn]

Ok, yes, so we can use this PR for the documentation, is that what you are hinting at?

No, I just referred to the comment by @sfinkens:

I think the zip file example would be very useful in the docs of pygac or satpy!

So if you tell me where to put the example, I would add it there.

[sfinkens]

What about adding that zip file example to the FSFile class documentation? If it has settled a bit, we can make it available in the main documentation.

[carloshorn]

What about adding that zip file example to the FSFile class documentation? If it has settled a bit, we can make it available in the main documentation.

Sounds good, I would combine it with a branch that also tackles #1475.

[gerritholl]

@carloshorn What is the status of this PR? Is it still relevant?