logging/logadmin: TestListMetrics failed

[flaky-bot[bot]]

Note: #9284 was also for this test, but it was closed more than 10 days ago. So, I didn't mark it flaky.

---

commit: f1bca4cde57239cd3c606a1566e83a7d7f5e7953 buildURL: Build Status, Sponge status: failed

[wwilfinger]

https://github.com/hashicorp/terraform-provider-google/blob/main/google/services/logging/resource_logging_bucket_config.go#L138-L140

Perhaps this works for other parent types like "folder" and "organization", but doesn't work as well with "project"? I don't have folder or orgs set up for GCP so I cannot easily test that theory.

[edwardmedia]

b/299177196

[sdykhner-twc]

@wwilfinger

I have the same issue using Terraform 1.4.6 and hashicorp/google provider 4.65.2

I was able to "fix" it via following steps (my infrastructure uses remote state):

1. Run terraform state pull > example.tfstate to retrieve the current state of your infrastructure.
2. Open the pulled file. Increase "serial" number of the state. This is needed to be able to push the state back. Then find your google_logging_project_bucket_config.default resource in the file (you may be able to find it by searching for "type": "google_logging_project_bucket_config", "google_logging_project_bucket_config" or resource name) and edit its "project" field by removing project/ from the value. Save the changes.
3. Run terraform state push example.tfstate to push new state to the backend.

If you do not store your backend remotely, I assume you just have to edit local state file.

Sure, it's just a workaround, but I hope it helps if you need to solve the issue "now" and proceed with your work.

[pengq-google]

Thanks for the comments. We are looking into it and will update as soon as possible. Comment here to keep the issue open.

[pengq-google]

I tried to reproduce this issue today but my import works at intended by using hashicorp/google provider 4.75.0

I used the commands to try to reproduce the issue:

terraform init -upgrade
rm terraform.tfstate.backup
rm terraform.tfstate
terraform import google_logging_project_bucket_config.{my_bucket} projects/{my_project_id}/locations/global/buckets/{my_bucket_id}

If this issue re-occurs, do you mind share the log here with TF_LOG=DEBUG?

TF_LOG=DEBUG terraform import google_logging_project_bucket_config.{your_bucket} projects/{project_id}/locations/{location_id}/buckets/{bucket_id}

Thanks!

[wwilfinger]

My original post was opened against the provider v4.81.0. I reproduced this morning with both the latest v4.84.0 and the v4.75.0 you used.

To be clear, the import is successful but on subsequent plan there is a destroy and replace diff because of a diff in the "project" argument of the resource.

Relevant bit of debug log with the project id and bucket name replaced to be generic

2023-10-03T09:50:28.059-0500 [WARN]  Provider "registry.terraform.io/hashicorp/google" produced an unexpected new value for google_logging_project_bucket_config.workload during refresh.
      - .enable_analytics: was null, but now cty.False
      - .lifecycle_state: was null, but now cty.StringVal("ACTIVE")
      - .locked: was null, but now cty.False
      - .name: was null, but now cty.StringVal("projects/example-project-id/locations/global/buckets/example-bucket-name")
      - .retention_days: was null, but now cty.NumberIntVal(30)
      - .description: was null, but now cty.StringVal("")

I'm happy to provide the full debug logs, but I do not want to try to obfuscate them and post them on the internet forever. You can email me at my github handle at google mail and I can send them over. Appreciate it.

[pengq-google]

Thank you for your feedback! I just sent you an email and re-open the ticket in our system. I'll keep working on it.

[pdecat]

Hi @wwilfinger , are you actually using a google_project datasource to feed the project field of the google_logging_project_bucket_config resource?

I've just discovered that it puts the projects/ prefix in its id attribute.

[wwilfinger]

are you actually using a google_project datasource to feed the project field of the google_logging_project_bucket_config resource?

The project id as a string literal right on the resource, like project = "example-project-id", reproduces.

[pdecat]

The Create function of the google_logging_project_bucket_config resource is supposed to automatically import the resource if it exists, and create it if it doesn't: https://github.com/terraform-providers/terraform-provider-google/blob/3eeed812d3bbeffb050002db14ffaefb8db3ea17/google/services/logging/resource_logging_project_bucket_config.go#L141

If you remove the google_logging_project_bucket_config.default resource from your state, what does an apply do?

Regarding the manual import behavior, I believe it is coming from this generic function: https://github.com/terraform-providers/terraform-provider-google/blob/3e00cfbafa1ef13ebfd7e039bcec77407b824bfc/google/services/logging/resource_logging_bucket_config.go#L142

And the 404 error happening when passing the projects/ prefix in the project argument is coming from this part where the resourceLoggingBucketConfigCreate() function also adds it a second time: https://github.com/terraform-providers/terraform-provider-google/blob/3e00cfbafa1ef13ebfd7e039bcec77407b824bfc/google/services/logging/resource_logging_bucket_config.go#L213

[pengq-google]

I reproduced this error on my laptop:

  # google_logging_project_bucket_config.tf_bucket_beta must be replaced
-/+ resource "google_logging_project_bucket_config" "tf_bucket_beta" {
        bucket_id        = "tf_bucket_beta"
        description      = "tf_bucket_beta"
      - enable_analytics = false -> null
      ~ id               = "projects/<my-project>/locations/global/buckets/tf_bucket_beta" -> (known after apply)
      ~ lifecycle_state  = "ACTIVE" -> (known after apply)
        location         = "global"
      - locked           = false -> null
      ~ name             = "projects/<my-project>/locations/global/buckets/tf_bucket_beta" -> (known after apply)
      ~ project          = "projects/<my-project>" -> "<my-project>" # forces replacement
        retention_days   = 30
    }

with config

resource "google_logging_project_bucket_config" "tf_bucket_beta" {
  provider    = google-beta
  project     = google_project.project0.project_id
  location    = "global"
  description = "tf_bucket_beta"
  bucket_id   = "tf_bucket_beta"

[pengq-google]

changed my config from

resource "google_logging_project_bucket_config" "tf_bucket_beta" {
  provider    = google-beta
  project     = google_project.project0.project_id
  location    = "global"
  description = "tf_bucket_beta"
  bucket_id   = "tf_bucket_beta"
}

to

resource "google_logging_project_bucket_config" "tf_bucket_beta" {
  provider    = google-beta
  project     = "projects/${google_project.project0.project_id}"
  location    = "global"
  description = "tf_bucket_beta"
  bucket_id   = "tf_bucket_beta"
}

solved the issue.

I am not sure why this happened or how long it has been like this. Thank you for raise the issue. Now I suspect it is because only resourceLoggingProjectBucketConfigAcquireOrCreate takes in projectBucketConfigID, while resourceLoggingProjectBucketConfigUpdate doesn't. https://github.com/hashicorp/terraform-provider-google/blob/0a8915bf4b03100573aec7caa8b3491567efa028/google/services/logging/resource_logging_project_bucket_config.go#L113C6-L113C27

Will try to modify the code to see if this issue is fixed. Before I confirming the root cause and fixing it, please using project = "projects/${google_project.project0.project_id}" to work around.

[pengq-google]

pdecat

Thanks for digging into this!:)

[pengq-google]

@pdecat Sorry I saw your confused emoji. I am not sure whether you are confused by me @ you, or confused by what I said above. I didn't have time to look into this issue (but it's always on my mind!) until last night, so I just spent some time on reading the code base, and left some comments. Then I noticed that you've already looked at the code and gave us advice. I just want to say thank you for your help! ^_^

[pdecat]

@pdecat Sorry I saw your confused emoji.

Oops, it was a mis-click, I meant :heart:

[manzomanze]

Do you have any updates on this issue? we encountered it today

[MaStFAU]

also important to mention that the docs on https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/logging_project_bucket_config are misleading because it states These buckets cannot be removed so deleting this resource will remove the bucket config from your terraform state but will leave the logging bucket unchanged. which I find misleading. I guess it is only talking about the _Required and _Default buckets but custom buckets will be deleted. And this is problematic with the given issue as TF requires a replacement but deleting the bucket is not done instantly as mentioned above.

[andrewnicolalde]

Encountered this today, seems it's still not fixed :/

[pengq-google]

Encountered this today, seems it's still not fixed :/

< May I ask what Terraform version you are using? Maybe upgrade will solve it? ^^

[andrewnicolalde]

@pengq-google I'm on terraform provider version 5.44.2, admittedly 4 months old.

[pengq-google]

Discussed with @andrewnicolalde This is fixed in https://github.com/GoogleCloudPlatform/magic-modules/pull/12574 sorry for forgetting to mark this issue as fixed.

[pengq-google]

_< seems I have no rights to mark this as fixed directly...

[Xilis]

I think this still exists, using provider v6.35.0

resource "google_logging_project_bucket_config" "default_bucket" {
  location  = "global"
  bucket_id = "_Default"

  # project   = "projects/my-project" -> works fine if this is used instead
  project = my-project
}

import {
  to = google_logging_project_bucket_config.default_bucket
  id = "projects/my-project/locations/global/buckets/_Default"
}

Leads to

  # google_logging_project_bucket_config.default_bucket must be replaced
  # (imported from "projects/my-project/locations/global/buckets/_Default")
  # Warning: this will destroy the imported resource

...
      ~ project          = "projects/my-project" -> "my-project" # forces replacement