magic-modules icon indicating copy to clipboard operation
magic-modules copied to clipboard

Refactor some Compute tests to use the `context` function signature

Open SarahFrench opened this issue 1 year ago • 24 comments

I was looking at https://github.com/hashicorp/terraform-provider-google/issues/17838 and realised that changing the tests there are tough because there are lots of arguments passed into config and formatting of the config string is hard to follow.

To enable those tests being edited, I decided to refactor some tests so that configs are generated using a config map instead of multiple arguments.

Note: I don't have the ability to run these tests in the GCP projects available to me.

Release Note Template for Downstream PRs (will be copied)


SarahFrench avatar Jun 05 '24 15:06 SarahFrench

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google-beta provider: Diff ( 1 file changed, 366 insertions(+), 309 deletions(-))

modular-magician avatar Jun 05 '24 15:06 modular-magician

Tests analytics

Total tests: 0 Passed tests: 0 Skipped tests: 0 Affected tests: 0

Click here to see the affected service packages
  • compute

$\textcolor{red}{\textsf{Errors occurred during REPLAYING mode. Please fix them to complete your PR.}}$ View the build log

modular-magician avatar Jun 05 '24 16:06 modular-magician

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google-beta provider: Diff ( 1 file changed, 107 insertions(+), 60 deletions(-))

modular-magician avatar Jun 05 '24 16:06 modular-magician

Tests analytics

Total tests: 0 Passed tests: 0 Skipped tests: 0 Affected tests: 0

Click here to see the affected service packages
  • compute

$\textcolor{red}{\textsf{Errors occurred during REPLAYING mode. Please fix them to complete your PR.}}$ View the build log

modular-magician avatar Jun 05 '24 16:06 modular-magician

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google-beta provider: Diff ( 1 file changed, 111 insertions(+), 63 deletions(-))

modular-magician avatar Jun 05 '24 16:06 modular-magician

Tests analytics

Total tests: 952 Passed tests: 877 Skipped tests: 74 Affected tests: 1

Click here to see the affected service packages
  • compute

Action taken

Found 1 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests
TestAccComputeInstanceNetworkIntefaceWithSecurityPolicy

Get to know how VCR tests work

modular-magician avatar Jun 05 '24 16:06 modular-magician

$\textcolor{red}{\textsf{Tests failed during RECORDING mode:}}$ TestAccComputeInstanceNetworkIntefaceWithSecurityPolicy[Error message] [Debug log]

$\textcolor{red}{\textsf{Please fix these to complete your PR.}}$ View the build log or the debug log for each test

modular-magician avatar Jun 05 '24 17:06 modular-magician

Current issues in TestAccComputeInstanceNetworkIntefaceWithSecurityPolicy

I ran a sweeper to address this one:

    vcr_utils.go:152: Step 1/2 error: Error running apply: exit status 1
        
        Error: Error creating NetworkEdgeSecurityService: googleapi: Error 400: Invalid value for field 'resource.region': ''. Region europe-west1 already has a Network Edge Security Service., invalid
        
          with google_compute_network_edge_security_service.edge_sec_service,
          on terraform_plugin_test.tf line 18, in resource "google_compute_network_edge_security_service" "edge_sec_service":
          18: resource "google_compute_network_edge_security_service" "edge_sec_service" {

c2560dc should address this one, it was due to a problem in the refactoring where I used reference paths instead of names

    vcr_utils.go:152: Step 1/6 error: Check failed: Check 2/2 error: Security Policy with name google_compute_region_security_policy.policyforinstance.self_link not present

SarahFrench avatar Jun 06 '24 15:06 SarahFrench

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google-beta provider: Diff ( 1 file changed, 115 insertions(+), 63 deletions(-))

modular-magician avatar Jun 06 '24 16:06 modular-magician

Tests analytics

Total tests: 953 Passed tests: 878 Skipped tests: 74 Affected tests: 1

Click here to see the affected service packages
  • compute

Action taken

Found 1 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests
TestAccComputeInstanceNetworkIntefaceWithSecurityPolicy

Get to know how VCR tests work

modular-magician avatar Jun 06 '24 17:06 modular-magician

$\textcolor{red}{\textsf{Tests failed during RECORDING mode:}}$ TestAccComputeInstanceNetworkIntefaceWithSecurityPolicy[Error message] [Debug log]

$\textcolor{red}{\textsf{Please fix these to complete your PR.}}$ View the build log or the debug log for each test

modular-magician avatar Jun 06 '24 17:06 modular-magician

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google-beta provider: Diff ( 1 file changed, 121 insertions(+), 64 deletions(-))

modular-magician avatar Jun 06 '24 18:06 modular-magician

Tests analytics

Total tests: 955 Passed tests: 880 Skipped tests: 74 Affected tests: 1

Click here to see the affected service packages
  • compute

Action taken

Found 1 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests
TestAccComputeInstanceNetworkIntefaceWithSecurityPolicy

Get to know how VCR tests work

modular-magician avatar Jun 06 '24 18:06 modular-magician

$\textcolor{red}{\textsf{Tests failed during RECORDING mode:}}$ TestAccComputeInstanceNetworkIntefaceWithSecurityPolicy[Error message] [Debug log]

$\textcolor{red}{\textsf{Please fix these to complete your PR.}}$ View the build log or the debug log for each test

modular-magician avatar Jun 06 '24 18:06 modular-magician

I'm going to run a sweeper to address the errors in the latest test run, then re-run the test

SarahFrench avatar Jun 07 '24 13:06 SarahFrench

/gcbrun

SarahFrench avatar Jun 07 '24 13:06 SarahFrench

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google-beta provider: Diff ( 1 file changed, 121 insertions(+), 64 deletions(-))

modular-magician avatar Jun 07 '24 13:06 modular-magician

Tests analytics

Total tests: 955 Passed tests: 880 Skipped tests: 74 Affected tests: 1

Click here to see the affected service packages
  • compute

Action taken

Found 1 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests
TestAccComputeInstanceNetworkIntefaceWithSecurityPolicy

Get to know how VCR tests work

modular-magician avatar Jun 07 '24 14:06 modular-magician

$\textcolor{red}{\textsf{Tests failed during RECORDING mode:}}$ TestAccComputeInstanceNetworkIntefaceWithSecurityPolicy[Error message] [Debug log]

$\textcolor{red}{\textsf{Please fix these to complete your PR.}}$ View the build log or the debug log for each test

modular-magician avatar Jun 07 '24 14:06 modular-magician

/gcbrun

SarahFrench avatar Jun 10 '24 18:06 SarahFrench

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google-beta provider: Diff ( 1 file changed, 121 insertions(+), 64 deletions(-))

modular-magician avatar Jun 10 '24 19:06 modular-magician

Tests analytics

Total tests: 956 Passed tests: 881 Skipped tests: 74 Affected tests: 1

Click here to see the affected service packages
  • compute

Action taken

Found 1 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests
TestAccComputeInstanceNetworkIntefaceWithSecurityPolicy

Get to know how VCR tests work

modular-magician avatar Jun 10 '24 19:06 modular-magician

$\textcolor{red}{\textsf{Tests failed during RECORDING mode:}}$ TestAccComputeInstanceNetworkIntefaceWithSecurityPolicy[Error message] [Debug log]

$\textcolor{red}{\textsf{Please fix these to complete your PR.}}$ View the build log or the debug log for each test

modular-magician avatar Jun 10 '24 19:06 modular-magician

👆 sweeper stopped the test erroring out due to quota, need to act on them

SarahFrench avatar Jun 10 '24 20:06 SarahFrench

/gcbrun

SarahFrench avatar Jul 02 '24 14:07 SarahFrench

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google-beta provider: Diff ( 1 file changed, 121 insertions(+), 64 deletions(-))

modular-magician avatar Jul 02 '24 14:07 modular-magician

Tests analytics

Total tests: 965 Passed tests: 891 Skipped tests: 73 Affected tests: 1

Click here to see the affected service packages
  • compute

Action taken

Found 1 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests
  • TestAccComputeInstanceNetworkIntefaceWithSecurityPolicy

Get to know how VCR tests work

modular-magician avatar Jul 02 '24 15:07 modular-magician

$\textcolor{red}{\textsf{Tests failed during RECORDING mode:}}$ TestAccComputeInstanceNetworkIntefaceWithSecurityPolicy[Error message] [Debug log]

$\textcolor{red}{\textsf{Errors occurred during RECORDING mode. Please fix them to complete your PR.}}$

View the build log or the debug log for each test

modular-magician avatar Jul 02 '24 15:07 modular-magician

TestAccComputeInstanceNetworkIntefaceWithSecurityPolicy__two_nics_two_access_configs_update_remove_access_config

  | Error: Error waiting for Deleting RegionSecurityPolicy: The security_policy resource 'projects/ci-test-project-188019/regions/europe-west1/securityPolicies/tf-test-policyddosprotection-sl3vfk18a6' is already being used by 'projects/ci-test-project-188019/regions/europe-west1/networkEdgeSecurityServices/tf-test-edgesecservice-sl3vfk18a6'

SarahFrench avatar Jul 02 '24 16:07 SarahFrench

Hey! I'm closing this PR as a part of a cleanup of older inactive PRs, using a threshold of PRs in "Draft" last updated over a month ago. This doesn't represent rejection of the change, and feel free to comment for me to reopen it if you plan to pick it back up, or feel free to start a new PR with the same changes in the future.

rileykarson avatar Sep 06 '24 19:09 rileykarson