magic-modules icon indicating copy to clipboard operation
magic-modules copied to clipboard

Address dependency issues in TestAccFirestoreField_* tests

Open SarahFrench opened this issue 1 year ago • 23 comments

Adding an additional delay in TestAccFirestoreField_* tests to address an 100% error occurring since late Jan:

Screenshot 2024-02-09 at 16 43 41

------- Stdout: -------
=== RUN   TestAccFirestoreField_firestoreFieldUpdateAddIndexExample
=== PAUSE TestAccFirestoreField_firestoreFieldUpdateAddIndexExample
=== CONT  TestAccFirestoreField_firestoreFieldUpdateAddIndexExample
    testing_new.go:84: Error running post-test destroy, there may be dangling resources: googleapi: Error 403: Cloud Firestore API has not been used in project tf-test3abifux6ev before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/firestore.googleapis.com/overview?project=tf-test3abifux6ev then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.
        Details:
        [
          {
            "@type": "type.googleapis.com/google.rpc.Help",
            "links": [
              {
                "description": "Google developers console API activation",
                "url": "https://console.developers.google.com/apis/api/firestore.googleapis.com/overview?project=tf-test3abifux6ev
              }
            ]
          },
          {
            "@type": "type.googleapis.com/google.rpc.ErrorInfo",
            "domain": "googleapis.com",
            "metadata": {
              "consumer": "projects/tf-test3abifux6ev",
              "service": "firestore.googleapis.com"
            },
            "reason": "SERVICE_DISABLED"
          }
        ]
--- FAIL: TestAccFirestoreField_firestoreFieldUpdateAddIndexExample (1163.75s)
FAIL

Release Note Template for Downstream PRs (will be copied)


SarahFrench avatar Feb 09 '24 16:02 SarahFrench

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

Terraform GA: Diff ( 1 file changed, 11 insertions(+), 3 deletions(-)) Terraform Beta: Diff ( 1 file changed, 11 insertions(+), 3 deletions(-))

modular-magician avatar Feb 09 '24 16:02 modular-magician

Tests analytics

Total tests: 17 Passed tests: 17 Skipped tests: 0 Affected tests: 0

Click here to see the affected service packages
  • firestore

$\textcolor{green}{\textsf{All tests passed!}}$ View the build log

modular-magician avatar Feb 09 '24 17:02 modular-magician

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

Terraform GA: Diff ( 1 file changed, 12 insertions(+), 4 deletions(-)) Terraform Beta: Diff ( 1 file changed, 12 insertions(+), 4 deletions(-))

modular-magician avatar Feb 13 '24 11:02 modular-magician

Tests analytics

Total tests: 17 Passed tests: 17 Skipped tests: 0 Affected tests: 0

Click here to see the affected service packages
  • firestore

$\textcolor{green}{\textsf{All tests passed!}}$ View the build log

modular-magician avatar Feb 13 '24 11:02 modular-magician

Ah, I just realised that this permission error is coming at the end of the test during cleanup. I had assumed the problem was enabling the service at the start of the test.

SarahFrench avatar Feb 13 '24 12:02 SarahFrench

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

Terraform GA: Diff ( 1 file changed, 4 insertions(+), 4 deletions(-)) Terraform Beta: Diff ( 1 file changed, 4 insertions(+), 4 deletions(-))

modular-magician avatar Feb 13 '24 12:02 modular-magician

Tests analytics

Total tests: 17 Passed tests: 17 Skipped tests: 0 Affected tests: 0

Click here to see the affected service packages
  • firestore

$\textcolor{green}{\textsf{All tests passed!}}$ View the build log

modular-magician avatar Feb 13 '24 12:02 modular-magician

<Comment removed because I'm being misleading, haha>

SarahFrench avatar Feb 13 '24 17:02 SarahFrench

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

Terraform GA: Diff ( 1 file changed, 8 insertions(+), 4 deletions(-)) Terraform Beta: Diff ( 1 file changed, 8 insertions(+), 4 deletions(-))

modular-magician avatar Feb 13 '24 17:02 modular-magician

Tests analytics

Total tests: 17 Passed tests: 17 Skipped tests: 0 Affected tests: 0

Click here to see the affected service packages
  • firestore

$\textcolor{green}{\textsf{All tests passed!}}$ View the build log

modular-magician avatar Feb 13 '24 17:02 modular-magician

Order of events in the test is:

  • 'Deletion' of firestore field resource - it's not deletable but is altered via PATCH request
  • Deletion of DB is skipped: "Firestore database "(default)" deletion_policy is not set to 'DELETE', skipping deletion"
  • Firestore service is disabled
  • Project is deleted
  • Then, there's a random GET request for the firestore field that triggers the 403

The GET request makes sense if the field cannot be deleted and the data base also isn't deleted. Terraform presumably wants to refresh state.

SarahFrench avatar Feb 13 '24 19:02 SarahFrench

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

Terraform GA: Diff ( 2 files changed, 10 insertions(+), 4 deletions(-)) Terraform Beta: Diff ( 2 files changed, 10 insertions(+), 4 deletions(-))

modular-magician avatar Feb 13 '24 19:02 modular-magician

Tests analytics

Total tests: 17 Passed tests: 17 Skipped tests: 0 Affected tests: 0

Click here to see the affected service packages
  • firestore

$\textcolor{green}{\textsf{All tests passed!}}$ View the build log

modular-magician avatar Feb 13 '24 19:02 modular-magician

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

Terraform GA: Diff ( 2 files changed, 12 insertions(+), 4 deletions(-)) Terraform Beta: Diff ( 2 files changed, 12 insertions(+), 4 deletions(-))

modular-magician avatar Feb 14 '24 12:02 modular-magician

Tests analytics

Total tests: 17 Passed tests: 17 Skipped tests: 0 Affected tests: 0

Click here to see the affected service packages
  • firestore

$\textcolor{green}{\textsf{All tests passed!}}$ View the build log

modular-magician avatar Feb 14 '24 12:02 modular-magician

Ah the problem appears to be that the CheckDestroy for the test isn't well equipped for when the test creates a new separate project.

We could use the useOwnProject bool to control which function is used to check if the project is destroyed

SarahFrench avatar Feb 14 '24 14:02 SarahFrench

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

Terraform GA: Diff ( 3 files changed, 19 insertions(+), 2 deletions(-)) Terraform Beta: Diff ( 3 files changed, 19 insertions(+), 2 deletions(-))

modular-magician avatar Feb 28 '24 13:02 modular-magician

Tests analytics

Total tests: 17 Passed tests: 17 Skipped tests: 0 Affected tests: 0

Click here to see the affected service packages
  • firestore

$\textcolor{green}{\textsf{All tests passed!}}$ View the build log

modular-magician avatar Feb 28 '24 13:02 modular-magician

This build in TeamCity runs TestAccFirestoreField_firestoreFieldUpdateAddIndexExample against this PR: https://hashicorp.teamcity.com/buildConfiguration/TerraformProviders_GoogleCloud_GOOGLE_MMUPSTREAMTESTS_GOOGLE_PACKAGE_FIRESTORE/108307?mode=builds&buildTab=overview&hideTestsFromDependencies=false&hideProblemsFromDependencies=false

SarahFrench avatar Feb 28 '24 13:02 SarahFrench

Test passed!

SarahFrench avatar Feb 28 '24 14:02 SarahFrench

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

Terraform GA: Diff ( 2 files changed, 16 insertions(+), 2 deletions(-)) Terraform Beta: Diff ( 2 files changed, 16 insertions(+), 2 deletions(-))

modular-magician avatar Feb 28 '24 14:02 modular-magician

Tests analytics

Total tests: 17 Passed tests: 17 Skipped tests: 0 Affected tests: 0

Click here to see the affected service packages
  • firestore

$\textcolor{green}{\textsf{All tests passed!}}$ View the build log

modular-magician avatar Feb 28 '24 14:02 modular-magician