Unable to authenticate with gcppostgres while in Terraform Cloud

[MrPickles]

Hey there. I'm looking to manage an existing postgres instance that lives in GCP. Everything works fine when I apply changes from a local machine, but I'm running into trouble when applying the same changes via Terraform Cloud. Specifically, running the apply from Terraform Cloud complains because it doesn't have the application default credentials.

The documentation mentions that you need to set the GOOGLE_APPLICATION_CREDENTIALS environment variable to be the path to the credentials file. However, to my knowledge, there's no way to create or store a hard drive file with Terraform Cloud, meaning there's nothing for the GOOGLE_APPLICATION_CREDENTIALS to point to.

Are there any known workarounds for situations where you're working on Terraform Cloud or don't have direct hard drive access? Thanks in advance for the help.

Terraform Version

v0.14.11

Affected Resource(s)

• postgresql_grant_role

Expected Behavior

Running terraform apply from Terraform Cloud connects to postgres in GCP.

Actual Behavior

Running terraform apply from Terraform Cloud fails to authenticate with postgres because there's no path to the Google application credentials. (Note that this is only an issue for Terraform Cloud. Everything works fine from a local machine.)

Error: Error connecting to PostgreSQL server mygcpproject:myregion:mydbinstance (scheme: gcppostgres): gcppostgres open gcppostgres://terraform:imagineapasswordhere@smygcpproject:myregion:mydbinstance:5432/postgres?fallback_application_name=Terraform+provider: google: could not find default credentials. See https://developers.google.com/accounts/docs/application-default-credentials for more information.

Steps to Reproduce

1. Create a postgres instance in GCP.
2. Use the postgres provider to connect to that DB instance. It should work fine on a local machine.
3. Use the same configs on Terraform Cloud. The apply fails because there's no good way to get the application default credentials.

Important Factoids

Here's a granular list representation of the previously-mentioned context:

• There's a postgres instance that lives in Google Cloud.
• Using the postgres provider works fine on a local machine.
• Using the same Terraform code on Terraform cloud runs into an error due to lack of application default credentials.
• The documentation mentioned setting the GOOGLE_APPLICATION_CREDENTIALS environment variable. While it's possible to set the environment variable itself on Terraform Cloud, that variable still needs to point to the credentials json file. There's no way to create that file on Terraform Cloud.

References

• https://github.com/cyrilgdn/terraform-provider-postgresql/issues/100 may be related. IAM authentication might be a possible fix to this issue.

[Deiz]

Ran into this one too, as a fellow Terraform Cloud user leveraging Hashicorp's remote executors.

To unblock myself, I forked the provider and threw in a simplistic shim that copies GOOGLE_CREDENTIALS (JSON, in-memory) to a temporary file and points GOOGLE_APPLICATION_CREDENTIALS at that file if the provider can't obtain application default credentials via other means. It's published at https://registry.terraform.io/providers/Deiz/postgresql/latest.

I'm only okay with this for the following reason, from Hashicorp's documentation:

Terraform Cloud is designed as an execution platform for Terraform, and can perform Terraform runs on its own disposable virtual machines.

That said, this feels like a missing Terraform Cloud feature more than a provider issue, so unless there's an appetite for it (@cyrilgdn - thoughts?) I won't attempt to push https://github.com/Deiz/terraform-provider-postgresql/commit/17109e15950faaa7ba9d05cf4e9f73cbc47edadb upstream.

[jeremylvln]

That said, this feels like a missing Terraform Cloud feature more than a provider issue, so unless there's an appetite for it (@cyrilgdn - thoughts?) I won't attempt to push Deiz@17109e1 upstream.

It will be highly appreciated having this feature in this provider directly!

[LiuVII]

I think it'd be even better to make this provider utilize GOOGLE_CREDENTIALS var directly instead of writing/reading it from the file but I'd take even the workaround