go-cloud
go-cloud copied to clipboard
blob/gcsblob: Support unauthenticated credentials for public buckets
Is your feature request related to a problem? Please describe.
I'm attempting to write a tool to fetch public GCS objects (from the gs://knative-release
bucket) for nightly automation. Given that the buckets are public, I'd prefer not to supply any GCS credentials (and the API explorer seems to indicate that they are not required).
Describe the solution you'd like
Provide an option / additional URLOpener
and scheme for anonymous GCS usage. This could be a global method gcsblob.RequireCredentials(boolean)
or a separate module or gs-noauth://
URL scheme.
Describe alternatives you've considered
Re-registering the gs://
handler in the DefaultURlMux
with a custom opener with my own gcp.HTTPClient
.
Additional context
I'm happy to provide a patch with whichever pattern is of interest. (My thought would be the RequireCredentials
path would be cleanest, and would hook the lazyCredsOpener
, rather than requiring re-registering the prefix.)
If I understand correctly, gcsblob.OpenBucket
handles this already, because it takes a gcp.HTTPClient
. So does the URLOpener
, since it also has a gcp.HTTPClient
field. So this is just about the default URLOpener
? If so, a PR adding a separate scheme gs-noauth://
sounds fine. An alternative would be to use the existing scheme plus an environment variable similar to the other credential stuff, near https://github.com/google/go-cloud/blob/master/blob/gcsblob/gcsblob.go#L157, but a new scheme seems more obvious and more flexible.
My ideal would actually be to have a client which starts with no auth and then retries with auth on 403 or 404 errors, but that looks to be substantially more complicated.
I'm hoping to make the tool usable in both nightly automation and for end-user use, so the environment variable seems like a good compromise. I'll prototype that and send a PR shortly.
Another way to configure this would be to allow for blank query param access_id=-
for just the bucket, the hyphen symbolizing it were intentionally omitted. (Option.GoogleAccessID
will eventually be obsoleted by SignBytes
.) That'll clear any fields meant for unrelated features.
OpenBucketURL
then would supply a client without credentials.
--- blob/gcsblob/gcsblob.go
+++ blob/gcsblob/gcsblob.go
@@ -225,1 +225,5 @@
+ client := o.Client
+ if opts.GoogleAccessID == "-" {
+ client = gcp.NewAnonymousHTTPClient(gcp.DefaultTransport())
+ }
+ return OpenBucket(ctx, client, u.Host, opts)
- return OpenBucket(ctx, o.Client, u.Host, opts)
(Fallback to credentials would then be swapping the bucket for another.)