blob/gcsblob: Support unauthenticated credentials for public buckets

[evankanderson]

Is your feature request related to a problem? Please describe.

I'm attempting to write a tool to fetch public GCS objects (from the gs://knative-release bucket) for nightly automation. Given that the buckets are public, I'd prefer not to supply any GCS credentials (and the API explorer seems to indicate that they are not required).

Describe the solution you'd like

Provide an option / additional URLOpener and scheme for anonymous GCS usage. This could be a global method gcsblob.RequireCredentials(boolean) or a separate module or gs-noauth:// URL scheme.

Describe alternatives you've considered

Re-registering the gs:// handler in the DefaultURlMux with a custom opener with my own gcp.HTTPClient.

Additional context

I'm happy to provide a patch with whichever pattern is of interest. (My thought would be the RequireCredentials path would be cleanest, and would hook the lazyCredsOpener, rather than requiring re-registering the prefix.)

[vangent]

If I understand correctly, gcsblob.OpenBucket handles this already, because it takes a gcp.HTTPClient. So does the URLOpener, since it also has a gcp.HTTPClient field. So this is just about the default URLOpener? If so, a PR adding a separate scheme gs-noauth:// sounds fine. An alternative would be to use the existing scheme plus an environment variable similar to the other credential stuff, near https://github.com/google/go-cloud/blob/master/blob/gcsblob/gcsblob.go#L157, but a new scheme seems more obvious and more flexible.

[evankanderson]

My ideal would actually be to have a client which starts with no auth and then retries with auth on 403 or 404 errors, but that looks to be substantially more complicated.

I'm hoping to make the tool usable in both nightly automation and for end-user use, so the environment variable seems like a good compromise. I'll prototype that and send a PR shortly.

[mark-kubacki]

Another way to configure this would be to allow for blank query param access_id=- for just the bucket, the hyphen symbolizing it were intentionally omitted. (Option.GoogleAccessID will eventually be obsoleted by SignBytes.) That'll clear any fields meant for unrelated features.

OpenBucketURL then would supply a client without credentials.

--- blob/gcsblob/gcsblob.go
+++ blob/gcsblob/gcsblob.go
@@ -225,1 +225,5 @@
+ client := o.Client
+ if opts.GoogleAccessID == "-" {
+   client = gcp.NewAnonymousHTTPClient(gcp.DefaultTransport())
+ }
+ return OpenBucket(ctx, client, u.Host, opts)
- return OpenBucket(ctx, o.Client, u.Host, opts)

(Fallback to credentials would then be swapping the bucket for another.)