Infer GCP projectId from the serviceAccount property instead of requiring it to be passed explicitly

[MichaelShoemaker]

Explain the bug

Creating GCP Bucket fails when using base64 encoded secret. If I mount my service account json file into the container and set the GOOGLE_APPLICATION_CREDENTIAL manually it works fine. However, when the json file is encoded using:

echo "SECRET_GCP_CRED=$(base64 -w 0 ./my_key/my-cred.json)" > .env_encoded

The task fails with the error:

2023-11-18 11:21:01.457 • Using service account: [email protected]
2023-11-18 11:21:02.777Creating bucket 'BucketInfo{name=kestra-test-bucket549}'
2023-11-18 11:21:02.783java.lang.NullPointerException: Required parameter project must be specified.
2023-11-18 11:21:02.783Required parameter project must be specified.
2023-11-18 11:21:02.783com.google.cloud.storage.StorageException: java.lang.NullPointerException: Required parameter project must be specified.
	at com.google.cloud.storage.StorageException.getStorageException(StorageException.java:101)
	at com.google.cloud.storage.StorageException.coalesce(StorageException.java:121)
	at com.google.cloud.storage.Retrying.run(Retrying.java:68)
	at com.google.cloud.storage.StorageImpl.run(StorageImpl.java:1515)
	at com.google.cloud.storage.StorageImpl.create(StorageImpl.java:128)
	at io.kestra.plugin.gcp.gcs.CreateBucket.run(CreateBucket.java:56)
	at io.kestra.plugin.gcp.gcs.CreateBucket.run(CreateBucket.java:15)
	at io.kestra.core.runners.Worker$WorkerThread.run(Worker.java:684)
Caused by: java.lang.NullPointerException: Required parameter project must be specified.
	at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:921)
	at com.google.api.client.util.Preconditions.checkNotNull(Preconditions.java:138)
	at com.google.api.services.storage.Storage$Buckets$Insert.<init>(Storage.java:1543)
	at com.google.api.services.storage.Storage$Buckets.insert(Storage.java:1518)
	at com.google.cloud.storage.spi.v1.HttpStorageRpc.create(HttpStorageRpc.java:346)
	at com.google.cloud.storage.StorageImpl.lambda$create$0(StorageImpl.java:130)
	at com.google.api.gax.retrying.DirectRetryingExecutor.submit(DirectRetryingExecutor.java:103)
	at com.google.cloud.RetryHelper.run(RetryHelper.java:76)
	at com.google.cloud.RetryHelper.runWithRetries(RetryHelper.java:50)
	at com.google.cloud.storage.Retrying.run(Retrying.java:65)
	... 5 more

My flow code is:

id: bucket-flow
namespace: dev
tasks:
  - id: "create_bucket"
    type: "io.kestra.plugin.gcp.gcs.CreateBucket"
    name: "kestra-test-bucket549"
    versioningEnabled: true
    # projectId: terraform-demo-397122
    serviceAccount: "{{ secret('GCP_CRED') }}"

When I uncomment to projectId and set the projectId it works with the base64 encoded json file. Decoding the file I see that the projectID is present which leads me to believe it may not be being parsed or passed correctly

root@641f8351815e:/app# decoded_creds=$(echo $SECRET_GCP_CRED | base64 -d) root@641f8351815e:/app# echo $decoded_creds { "type": "service_account", "project_id": "terraform-demo-397122", "private_key_id": "43ee0a9f9b......83e814bafd", "private_key": "-----BEGIN PRIVATE KEY-----\n......}

Environment Information

• Kestra Version: latest docker image. Unsure where to check this.
• Operating System and Java Version (if not using Kestra Docker image):