data "google_secret_manager_secret_version" alters binary content

[cghislai]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request.
• Please do not leave +1 or me too comments, they generate extra noise for issue followers and do not help prioritize the request.
• If you are interested in working on this issue or have submitted a pull request, please leave a comment.
• If an issue is assigned to a user, that user is claiming responsibility for the issue.
• Customers working with a Google Technical Account Manager or Customer Engineer can ask them to reach out internally to expedite investigation and resolution of this issue.

Terraform Version

Terraform v1.7.2 on linux_amd64

• provider registry.terraform.io/hashicorp/archive v2.4.2
• provider registry.terraform.io/hashicorp/google v5.16.0
• provider registry.terraform.io/hashicorp/googleworkspace v0.7.0
• provider registry.terraform.io/hashicorp/kubernetes v2.25.2
• provider registry.terraform.io/hashicorp/random v3.6.0
• provider registry.terraform.io/hashicorp/template v2.2.0

Affected Resource(s)

data source google_secret_manager_secret_version

Terraform Configuration

data "google_secret_manager_secret_version" "test" {
  secret = "test"
}

data "template_file" "test" {
  template =  base64encode(data.google_secret_manager_secret_version.test.secret_data)
}

Debug Output

No response

Expected Behavior

The binary content of the secret version uploaded on gcp secretmanager is preserved

eg;

Actual Behavior

The binary content of the secret version data is altered.

Steps to reproduce

1. Create a gcp secret version by uploading a binary file, for instance a zip archive
2. Get the secret data in a google_secret_manager_secret_version datasource
3. Obtain the binary data from terraform, for instance using the template provider like in the example above
4. terraform apply
5. terraform state show template_file.text; and compare the binary content

Important Factoids

I can get the binary data from a secret version using gcloud:

$  gcloud secrets versions access 4 --secret 'test'     --format='get(payload.data)' | tr '_-' '/+' 
UEsDBAoAAAAAAPNkVFgwTbbeFgAAABYAAAAKABwAdG1wL2JpbmFyeVVUCQADeo/UZYqP1GV1eAsAAQToAwAABOgDAABBCkIKMQoyCuKCrArOqQrCsMKwwrAKUEsBAh4DCgAAAAAA82RUWDBNtt4WAAAAFgAAAAoAGAAAAAAAAQAAAKSBAAAAAHRtcC9iaW5hcnlVVAUAA3qP1GV1eAsAAQToAwAABOgDAABQSwUGAAAAAAEAAQBQAAAAWgAAAAAA

The data is decodable into a zip file:

$  gcloud secrets versions access 4 --secret 'test'     --format='get(payload.data)' |   tr '_-' '/+'   | base64 -d > /tmp/binarycontent.zip
$ unzip -l /tmp/binarycontent.zip
Archive:  /tmp/binarycontent.zip
  Length      Date    Time    Name
---------  ---------- -----   ----
       22  2024-02-20 12:39   tmp/binary
---------                     -------
       22                     1 file

I can also get the text content of the secret using gcloud, and encode it using base64:

$  gcloud secrets versions access 4 --secret 'test' | base64 -w0 ;echo
UEsDBAoAAAAAAO+/vWRUWDBN77+977+9FgAAABYAAAAKABwAdG1wL2JpbmFyeVVUCQADeu+/ve+/vWXvv73vv73vv71ldXgLAAEE77+9AwAABO+/vQMAAEEKQgoxCjIK4oKsCs6pCsKwwrDCsApQSwECHgMKAAAAAADvv71kVFgwTe+/ve+/vRYAAAAWAAAACgAYAAAAAAABAAAA77+977+9AAAAAHRtcC9iaW5hcnlVVAUAA3rvv73vv71ldXgLAAEE77+9AwAABO+/vQMAAFBLBQYAAAAAAQABAFAAAABaAAAAAAA=

In this case, the binary content is altered, and I cannot recreate a readable zip archive from this. However, it is the exact same content that terraform provides in its secret_data field:

$  terraform state show template_file.test
# template_file.test:
data "template_file" "test" {
    id       = "55dd8970cd7d253ac9f2508db666748867714058cd38b8089e749eaa81d7ac6a"
    rendered = "UEsDBAoAAAAAAO+/vWRUWDBN77+977+9FgAAABYAAAAKABwAdG1wL2JpbmFyeVVUCQADeu+/ve+/vWXvv73vv73vv71ldXgLAAEE77+9AwAABO+/vQMAAEEKQgoxCjIK4oKsCs6pCsKwwrDCsApQSwECHgMKAAAAAADvv71kVFgwTe+/ve+/vRYAAAAWAAAACgAYAAAAAAABAAAA77+977+9AAAAAHRtcC9iaW5hcnlVVAUAA3rvv73vv71ldXgLAAEE77+9AwAABO+/vQMAAFBLBQYAAAAAAQABAFAAAABaAAAAAAA="
    template = (sensitive value)
}

So it seems that terraform interprets the secret data as a string, resulting in the corruption of binary data.

The gcloud secret version acess documentation mentions:

    To get the raw bytes, have Google Cloud CLI print the response as
    base64-encoded and decode:

        $ gcloud secrets versions access 123 --secret=my-secret \
            --format='get(payload.data)' | tr '_-' '/+' | base64 -d

References

No response

b/326473689

[roaks3]

@cghislai this looks like something we've come across before, where the data is being corrupted when treated as binary vs a string. Could you take a look at this comment and see if it helps you resolve? https://github.com/hashicorp/terraform-provider-google/issues/10835#issuecomment-1007774773

[cghislai]

@roaks3 its similar.

A new feature to expose the secret data as base64-encoded string in the "google_secret_manager_secret_version" and 'google_secret_manager_secret_version_access" datasources is probably required as well.

The resource 'google_secret_manager_secret_version' already has a flag to handle binary data:

is_secret_data_base64 - (Optional) If set to 'true', the secret data is expected to be base64-encoded string and would be sent as is.

However, for binary secrets created outside of terraform, the datasources have no way to obtain usable data.

[c2thorn]

labeling as an enhancement based on https://github.com/hashicorp/terraform-provider-google/issues/10835

[roaks3]

Just to clarify for the service team before forwarding: this appears to be missing behavior specifically for the datasources (not the resource).

[mostafamohajeri]

Hi,

Is there any update or workarounds for this? my use-case is basically the same:

data "google_secret_manager_secret_version" "my-jks-file" {
  project = "my-project"
  secret  = "my-gsm-secret"
}
 
resource "kubernetes_secret" "my-jks-file-in-k8s" {
  metadata {
    name      = "my-jks-file-in-k8s"
    namespace = "my-ns"
  }
 
  binary_data = {
    "keystore-jks" = data.google_secret_manager_secret_version.my-jks-file.secret_data
  }
}

using binary_data results in base64 decode error, and using data just puts mangled data (correct decodable by linux base64 -d weirdly but not a jks when decoded)

doing this works however just fine, and get a usable jks file:

gcloud secrets versions access latest --secret=my-gsm-secret \
            --format='get(payload.data)' | tr '_-' '/+' | base64 -d > file.jks

ps. didn't create another issue to not duplicate

thanks

[roaks3]

Yea, unfortunately I don't think we have a good workaround for this one. Once the data is available from the resource (either using outputs or TF state directly), it is already corrupt, and I'm not aware of a clean way to get it back into a usable format. I believe gcloud is the only workaround at the moment.

[sergiojoker11]

As a workaround, we base64 binaries before placing them in Secret Manager - this is not something the GCP UI does when using the file picker - then, the secret_data you get from the data block doesn't seem to be corrupted. In our case, the secret_dataa ends up in a kubernetes secret as follows:

resource "kubernetes_secret" "kafka_clusters_auth" {
  metadata {
    name = "kafka-clusters-auth"
  }
  binary_data = {
    "client.truststore.jks" = var.your_base64_encoded_binary
}

where binary_data from docs is: (Optional) A map base64 encoded map of the secret data.. Source: https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/resources/secret.html#binary_data

[melinath]

Note from triage: Keeping this an enhancement as a request for a new field that allows binary content to behave as expected, since it likely can't be handled in the same way as string fields.