Provide signed URLs

[remram44]

trafficstars

Similar to S3, GS provides signed URLs, allowing someone to access an object without having credentials (for example, letting a web browser download a specific object from a private bucket): https://cloud.google.com/storage/docs/access-control/signed-urls

s3fs provides this (S3FileSystem.url() and S3File.url()) but gcsfs does not (GCSFile.url() and GCSFileSystem.url() exist but don't sign). I think at the very least this difference is surprising.

[martindurant]

If you have a way to produced the signed version, that would be greatly appreciated.

[remram44]

Yes, it does not seem straightforward. I am going to need this, so should I manage to implement it, I will send a patch.

Navigating those cloud APIs is so confusing (which is why fsspec and gcsfs are so amazing!)

[aberres]

Reference implementations can be found in google.cloud.storage._signing.generate_signed_url_v2 and google.cloud.storage._signing.generate_signed_url_v4.

[remram44]

Thanks, that helps! Direct link: https://github.com/googleapis/google-cloud-python/blob/2001d7645b9eba6186e5de2565d18580fe85bfcf/storage/google/cloud/storage/_signing.py#L399

The good news is that gcsfs already depends on google-auth so we don't have to re-implement the Credentials stuff. The bad news is that google-cloud-storage is licensed under Apache-2.0 so I don't think we can re-use that code, we probably have to implement it from the spec.

[TomAugspurger]

Is there anything preventing us from using that directly, rather than reimplementing it?

[pmrowla]

It looks like this is actually already implemented? gcsfs.url() does not do signing but gcsfs.sign() does (and only sign() is defined in fsspec/filesystem_spec) https://github.com/fsspec/gcsfs/blob/378cae6f56f3dbca1b909d58f21a8d61ed5b0a45/gcsfs/core.py#L1581-L1606

[martindurant]

I'd be happy to consider aliasing url->sign, or with an appropriate kwarg. I think the naming comes from s3fs and upstream botocore, but the intent is indeed to have sign() be the standard.

[martindurant]

(or we can just close this issue)

[pmrowla]

Can't speak for the user that originally opened this issue, but for our purposes sign() is sufficient