Python bindings: add a osgeo.gdal_fsspec module that on import will register GDAL VSI file system handlers as fsspec AbstractFileSystem

[rouault]

This enables using GDAL virtual file systems with other libraries of the Python ecosystem that accept fsspec paths

"""Module exposing GDAL Virtual File Systems (VSI) as a "gdalvsi" fsspec implementation.

Importing "osgeo.gdal_fsspec" requires the Python "fsspec" (https://filesystem-spec.readthedocs.io/en/latest/) module to be available.

A generic "gdalvsi" fsspec protocol is available. All GDAL VSI file names must be simply prefixed with "gdalvsi://". For example:

• "gdalvsi://data/byte.tif" to access relative file "data/byte.tif"
• "gdalvsi:///home/user/byte.tif" to access absolute file "/home/user/byte.tif"
• "gdalvsi:///vsimem/byte.tif" (note the 3 slashes) to access VSIMem file "/vsimem/byte.tif"
• "gdalvsi:///vsicurl/https://example.com/byte.tif (note the 3 slashes) to access "https://example.com/byte.tif" through /vsicurl/ """

[sgillies]

@rouault can we do without so many new forms of identifiers? As a community, I think we want fewer ways to reference the same thing, not more ways to reference the same thing.

Like, I imagine the fsspec usage would be:

fs = fsspec.filesystem("gdalvsi")
fs.open("/vsicurl/https://example.com/foo.tif")

The existing /vsi*/ files could remain the one way to reference datasets for GDAL, if we want.

Is "vsizip://my.zip/foo.tif" only for fsspec or will it work with GDALOpen too?

[rouault]

can we do without so many new forms of identifiers?

I also had the same hesitation, and I would be OK with just having a single "gdalvsi" fsspec protocol taking GDAL paths as you suggest

Is "vsizip://my.zip/foo.tif" only for fsspec or will it work with GDALOpen too?

No that would be only for fsspec, if we keep the "sub-classed" vsiXXX fsspec protocol

[rouault]

Pull request simplified to register a single "gdalvsi" fsspec protocol as suggested by @sgillies

[coveralls]

coverage: 69.435% (-0.01%) from 69.449% when pulling 0b07ea7db3c95b57e779a4ef0c59d0b08bf9f552 on rouault:gdal_fsspec into 8d9a3976c8727fa3fe53c558a9a8d161b3b20f54 on OSGeo:master.