planetary-computer-sdk-for-python icon indicating copy to clipboard operation
planetary-computer-sdk-for-python copied to clipboard

[POC]: Added "pc" fsspec filesystem

Open TomAugspurger opened this issue 3 years ago • 0 comments
trafficstars

This adds a "pc" fsspec filesystem implementation, which lets us insert "pc::" in an fsspec URL and automatically sign it when loading it with an fsspec client.

The primary motivation is integration with fsspec's filesystem where users would need to call planetary_computer.sign in multiple places

  1. Once for loading the index JSON files
  2. Once for signing the reference filesystem templates

Which lets us replace this:

>>> result = xr.open_dataset(
...     fsspec.get_mapper(
...         "reference://",
...         fo=planetary_computer.sign(requests.get(planetary_computer.sign("https://deltaresreservoirssa.blob.core.windows.net/references/reservoirs/chirps.json")).json()),
...     ),
...     engine="zarr",
...     consolidated=False,
... )

With this:

>>> result = xr.open_dataset(
...     "pc::reference::pc::https://deltaresreservoirssa.blob.core.windows.net/references/reservoirs/CHIRPS.json",
...     engine="zarr",
...     consolidated=False,
... )

Still just a POC. I need to figure out

  1. Better tests.
  2. If there's a way to modifier the references earlier.

TomAugspurger avatar Aug 08 '22 14:08 TomAugspurger