Support passing credential blob inline in GCS

[kennytm]

Feature Request

Describe your feature request related problem:

Currently GCS requires a file to pass the credentials. Creating a file is not always possible, esp. in BR-in-SQL scenario.

Describe the feature you'd like:

Support passing the credentials directly as a URL parameter.

• Option A: percent-escaping — gcs://bucket/path?credentials-json=%7B%22access_id%22%3A%22your_access_key_id%22%2C%22secret_key%22%3A%22your_secret_access_key%22%7D
• Option B: base64-encoding — gcs://bucket/path?credentials-json=eyJhY2Nlc3NfaWQiOiJ5b3VyX2FjY2Vzc19rZXlfaWQiLCJzZWNyZXRfa2V5IjoieW91cl9zZWNyZXRfYWNjZXNzX2tleSJ9

Describe alternatives you've considered:

Rely entirely on #633.

Teachability, Documentation, Adoption, Migration Strategy:

[IANTHEREAL]

I prefer to add another argument in br and option in br via sql

[iamxy]

I prefer to provide a convenient solution for online encode, such as base64encode. So, Option B is good for me.

[kennytm]

If the user can create an HMAC key and set a default project, they can reuse the s3:// protocol as illustrated in https://cloud.google.com/storage/docs/migrating#migration-simple.

[kennytm]

The credentials blob is extremely long, see https://cloud.google.com/iam/docs/creating-managing-service-account-keys#creating_service_account_keys for a template. So option B looks like the only viable way.

[tennix]

I think we can just postpone this feature, passing credentials around is not secure. The secure way is to grant permission directly to the ec2 node (iam role) or gce node (service account).

For TiDBCloud product, we can expose the underlying node's iam role or service account, then users just grant permissions to the iam role or service account.