magic-modules icon indicating copy to clipboard operation
magic-modules copied to clipboard
verified publisher icon GoogleCloudPlatform

secretmanager: added `fetch_secret_data` to `google_secret_manager_secret_version` to be able to skip fetching the secret data

Open ramonvermeulen opened this issue 6 months ago • 2 comments

Closes https://github.com/hashicorp/terraform-provider-google/issues/23236

Release Note Template for Downstream PRs (will be copied)

See Write release notes for guidance.

secretmanager: added `fetch_secret_data` to `google_secret_manager_secret_version` to be able to skip fetching the secret data

ramonvermeulen avatar Jun 19 '25 08:06 ramonvermeulen

Local tests:

❮ make testacc TEST=./google/services/secretmanager TESTARGS='-run=TestAccDatasourceSecretManagerSecretVersion_'

TF_ACC=1 TF_SCHEMA_PANIC_ON_ERROR=1 go test ./google/services/secretmanager -v -run=TestAccDatasourceSecretManagerSecretVersion_ -timeout 240m -ldflags="-X=github.com/hashicorp/terraform-provider-google/version.ProviderVersion=acc"
=== RUN   TestAccDatasourceSecretManagerSecretVersion_basic
=== PAUSE TestAccDatasourceSecretManagerSecretVersion_basic
=== RUN   TestAccDatasourceSecretManagerSecretVersion_fetchSecretDataFalse
=== PAUSE TestAccDatasourceSecretManagerSecretVersion_fetchSecretDataFalse
=== RUN   TestAccDatasourceSecretManagerSecretVersion_latest
=== PAUSE TestAccDatasourceSecretManagerSecretVersion_latest
=== RUN   TestAccDatasourceSecretManagerSecretVersion_withBase64SecretData
=== PAUSE TestAccDatasourceSecretManagerSecretVersion_withBase64SecretData
=== CONT  TestAccDatasourceSecretManagerSecretVersion_basic
=== CONT  TestAccDatasourceSecretManagerSecretVersion_latest
=== CONT  TestAccDatasourceSecretManagerSecretVersion_withBase64SecretData
=== CONT  TestAccDatasourceSecretManagerSecretVersion_fetchSecretDataFalse
--- PASS: TestAccDatasourceSecretManagerSecretVersion_basic (11.98s)
--- PASS: TestAccDatasourceSecretManagerSecretVersion_fetchSecretDataFalse (12.24s)
--- PASS: TestAccDatasourceSecretManagerSecretVersion_withBase64SecretData (13.96s)
--- PASS: TestAccDatasourceSecretManagerSecretVersion_latest (16.09s)
PASS
ok      github.com/hashicorp/terraform-provider-google/google/services/secretmanager    16.832s

ramonvermeulen avatar Jun 19 '25 18:06 ramonvermeulen

Hello! I am a robot. Tests will require approval from a repository maintainer to run.

Googlers: For automatic test runs see go/terraform-auto-test-runs.

@hao-nan-li, a repository maintainer, has been assigned to review your changes. If you have not received review feedback within 2 business days, please leave a comment on this PR asking them to take a look.

You can help make sure that review is quick by doing a self-review and by running impacted tests locally.

github-actions[bot] avatar Jun 19 '25 18:06 github-actions[bot]

@melinath This PR has been waiting for review for 3 weekdays. Please take a look! Use the label disable-review-reminders to disable these notifications.

github-actions[bot] avatar Jun 24 '25 09:06 github-actions[bot]

@GoogleCloudPlatform/terraform-team @melinath This PR has been waiting for review for 1 week. Please take a look! Use the label disable-review-reminders to disable these notifications.

github-actions[bot] avatar Jun 26 '25 09:06 github-actions[bot]

Sorry, missed that this wasn't related to writeonly stuff. Reassigning since I'll be OOO next week.

melinath avatar Jun 27 '25 23:06 melinath

@modular-magician reassign-reviewer

melinath avatar Jun 27 '25 23:06 melinath

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 3 files changed, 80 insertions(+), 25 deletions(-)) google-beta provider: Diff ( 3 files changed, 80 insertions(+), 25 deletions(-))

modular-magician avatar Jun 27 '25 23:06 modular-magician

Tests analytics

Total tests: 40 Passed tests: 37 Skipped tests: 2 Affected tests: 1

Click here to see the affected service packages
  • secretmanager
#### Action taken
Found 1 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests
  • TestAccDatasourceSecretManagerSecretVersion_fetchSecretDataFalse

Get to know how VCR tests work

modular-magician avatar Jun 27 '25 23:06 modular-magician

🟢 Tests passed during RECORDING mode: TestAccDatasourceSecretManagerSecretVersion_fetchSecretDataFalse [Debug log]

🟢 No issues found for passed tests after REPLAYING rerun.

🟢 All tests passed!

View the build log or the debug log for each test

modular-magician avatar Jun 27 '25 23:06 modular-magician

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 3 files changed, 80 insertions(+), 25 deletions(-)) google-beta provider: Diff ( 3 files changed, 80 insertions(+), 25 deletions(-))

modular-magician avatar Jun 30 '25 17:06 modular-magician

Tests analytics

Total tests: 40 Passed tests: 38 Skipped tests: 2 Affected tests: 0

Click here to see the affected service packages
  • secretmanager
🟢 All tests passed!

View the build log

modular-magician avatar Jun 30 '25 17:06 modular-magician

@hao-nan-li

As discussed in the comments, I did some extra manual testing today.

1) Initial configuration (only masked the project id):

data "google_secret_manager_secret_version" "default" {
  project = "my-project-id"
  secret  = "magic-modules-test"
  version = 1
}

output "secret_data" {
  sensitive = true
  value = {
    secret_data = data.google_secret_manager_secret_version.default.secret_data
  }
}

output "secret_meta" {
  value = {
    secret_name          = data.google_secret_manager_secret_version.default.name
    secret_creation_time = data.google_secret_manager_secret_version.default.create_time
    secret_destroy_time  = data.google_secret_manager_secret_version.default.destroy_time
    secret_is_enabled    = data.google_secret_manager_secret_version.default.enabled
  }
}

2) Terraform version:

~/projects/work/personal/terraform_test_suite                                                                                                                                                    󱇶 default 
➜ terraform --version        
Terraform v1.12.2
on darwin_arm64
+ provider registry.terraform.io/hashicorp/google v6.40.0

3) Terraform apply:

➜ terraform apply                                                               
data.google_secret_manager_secret_version.default: Reading...
data.google_secret_manager_secret_version.default: Read complete after 0s [id=projects/XXXXXXXXXXXXXX/secrets/magic-modules-test/versions/1]

Changes to Outputs:
  ~ secret_data = (sensitive value)
  + secret_meta = {
      + secret_creation_time = "2025-07-01T06:34:46.723956Z"
      + secret_destroy_time  = null
      + secret_is_enabled    = true
      + secret_name          = "projects/XXXXXXXXXXXXXX/secrets/magic-modules-test/versions/1"
    }

You can apply this plan to save these new output values to the Terraform state, without changing any real infrastructure.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes


Apply complete! Resources: 0 added, 0 changed, 0 destroyed.

Outputs:

secret_data = <sensitive>
secret_meta = {
  "secret_creation_time" = "2025-07-01T06:34:46.723956Z"
  "secret_destroy_time" = tostring(null)
  "secret_is_enabled" = true
  "secret_name" = "projects/XXXXXXXXXXXXXX/secrets/magic-modules-test/versions/1"
}

4) Another apply directly after:

➜ terraform apply
data.google_secret_manager_secret_version.default: Reading...
data.google_secret_manager_secret_version.default: Read complete after 0s [id=projects/XXXXXXXXXXXXXX/secrets/magic-modules-test/versions/1]

No changes. Your infrastructure matches the configuration.

Terraform has compared your real infrastructure against your configuration and found no differences, so no changes are needed.

Apply complete! Resources: 0 added, 0 changed, 0 destroyed.

Outputs:

secret_data = <sensitive>
secret_meta = {
  "secret_creation_time" = "2025-07-01T06:34:46.723956Z"
  "secret_destroy_time" = tostring(null)
  "secret_is_enabled" = true
  "secret_name" = "projects/XXXXXXXXXXXXXX/secrets/magic-modules-test/versions/1"
}

5) look at terraform state file:

    ...
    "secret_data": {
      "value": {
        "secret_data": "hello world"
      },
      "type": [
        "object",
        {
          "secret_data": "string"
        }
      ],
      "sensitive": true
    },
    ...

6) Then I build the provider on the ramonvermeulen:ramon/23236-add-fetch-secret-data-flag-to-secret-version branch, and did another apply with dev_overrides:

provider_installation {
  dev_overrides {
      "hashicorp/google" = "/Users/ramon/go/bin/"
      "hashicorp/google-beta" = "/Users/ramon/go/bin/"
  }

  direct {}
}

~/projects/work/personal/terraform_test_suite                                                                                                                                                    󱇶 default 
➜ TF_CLI_CONFIG_FILE="$(pwd)/tf-dev-override.tfrc" terraform apply

╷
│ Warning: Provider development overrides are in effect
│ 
│ The following provider development overrides are set in the CLI configuration:
│  - hashicorp/google in /Users/ramon/go/bin
│  - hashicorp/google-beta in /Users/ramon/go/bin
│ 
│ The behavior may therefore not match any released version of the provider and applying changes may cause the state to become incompatible with published releases.
╵
data.google_secret_manager_secret_version.default: Reading...
data.google_secret_manager_secret_version.default: Read complete after 0s [id=projects/XXXXXXXXXXXXXX/secrets/magic-modules-test/versions/1]

No changes. Your infrastructure matches the configuration.

Terraform has compared your real infrastructure against your configuration and found no differences, so no changes are needed.

Apply complete! Resources: 0 added, 0 changed, 0 destroyed.

Outputs:

secret_data = <sensitive>
secret_meta = {
  "secret_creation_time" = "2025-07-01T06:34:46.723956Z"
  "secret_destroy_time" = tostring(null)
  "secret_is_enabled" = true
  "secret_name" = "projects/XXXXXXXXXXXXXX/secrets/magic-modules-test/versions/1"
}

7) Another look at terraform state file:

    ...
    "secret_data": {
      "value": {
        "secret_data": "hello world"
      },
      "type": [
        "object",
        {
          "secret_data": "string"
        }
      ],
      "sensitive": true
    },
    ...

8) Change configuration to put fetch_secret_data on false

data "google_secret_manager_secret_version" "default" {
  project           = "my-project-id"
  secret            = "magic-modules-test"
  version           = 1
  fetch_secret_data = false
}

9) Ran another terraform apply (this one detects a change, which is desired so asks for confirmation):

~/projects/work/personal/terraform_test_suite                                                                                                                                                    󱇶 default 
➜ TF_CLI_CONFIG_FILE="$(pwd)/tf-dev-override.tfrc" terraform apply

╷
│ Warning: Provider development overrides are in effect
│ 
│ The following provider development overrides are set in the CLI configuration:
│  - hashicorp/google in /Users/ramon/go/bin
│  - hashicorp/google-beta in /Users/ramon/go/bin
│ 
│ The behavior may therefore not match any released version of the provider and applying changes may cause the state to become incompatible with published releases.
╵
data.google_secret_manager_secret_version.default: Reading...
data.google_secret_manager_secret_version.default: Read complete after 0s [id=projects/XXXXXXXXXXXXXX/secrets/magic-modules-test/versions/1]

Changes to Outputs:
  ~ secret_data = (sensitive value)

You can apply this plan to save these new output values to the Terraform state, without changing any real infrastructure.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes


Apply complete! Resources: 0 added, 0 changed, 0 destroyed.

Outputs:

secret_data = <sensitive>
secret_meta = {
  "secret_creation_time" = "2025-07-01T06:34:46.723956Z"
  "secret_destroy_time" = tostring(null)
  "secret_is_enabled" = true
  "secret_name" = "projects/XXXXXXXXXXXXXX/secrets/magic-modules-test/versions/1"
}

10) Another look at the terraform state:

    ...
    "secret_data": {
      "value": {
        "secret_data": null
      },
      "type": [
        "object",
        {
          "secret_data": "string"
        }
      ],
      "sensitive": true
    },
    ...

So without explicitly setting fetch_secret_data to false it will not have effect on existing configurations.

ramonvermeulen avatar Jul 01 '25 07:07 ramonvermeulen

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 3 files changed, 81 insertions(+), 26 deletions(-)) google-beta provider: Diff ( 3 files changed, 81 insertions(+), 26 deletions(-))

modular-magician avatar Jul 01 '25 16:07 modular-magician

Tests analytics

Total tests: 40 Passed tests: 38 Skipped tests: 2 Affected tests: 0

Click here to see the affected service packages
  • secretmanager
🟢 All tests passed!

View the build log

modular-magician avatar Jul 01 '25 16:07 modular-magician