magic-modules icon indicating copy to clipboard operation
magic-modules copied to clipboard

Update CMEK usage in Bigquery tests

Open rileykarson opened this issue 1 year ago • 8 comments

Fixes https://github.com/hashicorp/terraform-provider-google/issues/19066

Handful of custodial changes here in cleaning up some conflicting global IAM grants:

  • Move towards key-level permissions as is done in this guide: https://cloud.google.com/bigquery/docs/customer-managed-encryption#assign_role
    • Remove bootstrapping of a project-level role as well
  • Rename the google_bigquery_connection kms example to sql_with_cmek to better reflect what it actually showcases. Source page is https://cloud.google.com/bigquery/docs/connect-to-sql#create-sql-connection
  • Cleaned up some inconsistent whitespace that was either 4-space or tabs (to 2-space) and moved around some fields that were not ordered as count > identity > other settings

Release Note Template for Downstream PRs (will be copied)

rileykarson avatar Oct 09 '24 03:10 rileykarson

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 4 files changed, 79 insertions(+), 76 deletions(-)) google-beta provider: Diff ( 4 files changed, 79 insertions(+), 76 deletions(-)) Open in Cloud Shell: Diff ( 4 files changed, 149 insertions(+))

modular-magician avatar Oct 09 '24 03:10 modular-magician

Tests analytics

Total tests: 136 Passed tests: 120 Skipped tests: 13 Affected tests: 3

Click here to see the affected service packages
  • bigquery
  • bigqueryconnection

Action taken

Found 3 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests
  • TestAccBigQueryJob_bigqueryJobCopyExample
  • TestAccBigQueryJob_bigqueryJobCopyTableReferenceExample
  • TestAccBigqueryConnectionConnection_bigqueryConnectionSqlWithCmekExample

Get to know how VCR tests work

modular-magician avatar Oct 09 '24 03:10 modular-magician

🟢 Tests passed during RECORDING mode: TestAccBigQueryJob_bigqueryJobCopyExample[Debug log] TestAccBigQueryJob_bigqueryJobCopyTableReferenceExample[Debug log]

🟢 No issues found for passed tests after REPLAYING rerun.


🔴 Tests failed during RECORDING mode: TestAccBigqueryConnectionConnection_bigqueryConnectionSqlWithCmekExample[Error message] [Debug log]

🔴 Errors occurred during RECORDING mode. Please fix them to complete your PR.

View the build log or the debug log for each test

modular-magician avatar Oct 09 '24 03:10 modular-magician

This should work right now (we're no longer fighting over the permission) but still has a global static resource for the kms/sql_with_cmek test case. I may take a different approach- I'm undecided because it doesn't really matter right now while there's only one user (the other two tests use their own KMS keys and not the global static bootstrapped one).

rileykarson avatar Oct 09 '24 03:10 rileykarson

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 4 files changed, 79 insertions(+), 76 deletions(-)) google-beta provider: Diff ( 4 files changed, 79 insertions(+), 76 deletions(-)) Open in Cloud Shell: Diff ( 4 files changed, 149 insertions(+))

modular-magician avatar Oct 09 '24 03:10 modular-magician

Tests analytics

Total tests: 136 Passed tests: 122 Skipped tests: 13 Affected tests: 1

Click here to see the affected service packages
  • bigquery
  • bigqueryconnection

Action taken

Found 1 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests
  • TestAccBigqueryConnectionConnection_bigqueryConnectionSqlWithCmekExample

Get to know how VCR tests work

modular-magician avatar Oct 09 '24 03:10 modular-magician

🟢 Tests passed during RECORDING mode: TestAccBigqueryConnectionConnection_bigqueryConnectionSqlWithCmekExample[Debug log]

🟢 No issues found for passed tests after REPLAYING rerun.


🟢 All tests passed!

View the build log or the debug log for each test

modular-magician avatar Oct 09 '24 04:10 modular-magician

BootstrapPSARole applies the permission at the project level. Is it necessary to apply it at the key level?

Not if we applied it at the project level, but I removed that grant so now we need to apply it at the key level.

rileykarson avatar Oct 09 '24 22:10 rileykarson

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 4 files changed, 79 insertions(+), 76 deletions(-)) google-beta provider: Diff ( 4 files changed, 79 insertions(+), 76 deletions(-)) Open in Cloud Shell: Diff ( 4 files changed, 149 insertions(+))

modular-magician avatar Oct 22 '24 22:10 modular-magician

Tests analytics

Total tests: 137 Passed tests: 123 Skipped tests: 13 Affected tests: 1

Click here to see the affected service packages
  • bigquery
  • bigqueryconnection

Action taken

Found 1 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests
  • TestAccBigqueryConnectionConnection_bigqueryConnectionSqlWithCmekExample

Get to know how VCR tests work

modular-magician avatar Oct 22 '24 22:10 modular-magician

🟢 Tests passed during RECORDING mode: TestAccBigqueryConnectionConnection_bigqueryConnectionSqlWithCmekExample [Debug log]

🟢 No issues found for passed tests after REPLAYING rerun.


🟢 All tests passed!

View the build log or the debug log for each test

modular-magician avatar Oct 22 '24 22:10 modular-magician

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 4 files changed, 79 insertions(+), 76 deletions(-)) google-beta provider: Diff ( 4 files changed, 79 insertions(+), 76 deletions(-)) Open in Cloud Shell: Diff ( 4 files changed, 149 insertions(+))

modular-magician avatar Oct 22 '24 23:10 modular-magician

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 4 files changed, 79 insertions(+), 76 deletions(-)) google-beta provider: Diff ( 4 files changed, 79 insertions(+), 76 deletions(-)) Open in Cloud Shell: Diff ( 4 files changed, 149 insertions(+))

modular-magician avatar Oct 22 '24 23:10 modular-magician

Tests analytics

Total tests: 137 Passed tests: 124 Skipped tests: 13 Affected tests: 0

Click here to see the affected service packages
  • bigquery
  • bigqueryconnection

🟢 All tests passed!

View the build log

modular-magician avatar Oct 22 '24 23:10 modular-magician

Tests analytics

Total tests: 137 Passed tests: 124 Skipped tests: 13 Affected tests: 0

Click here to see the affected service packages
  • bigquery
  • bigqueryconnection

🟢 All tests passed!

View the build log

modular-magician avatar Oct 22 '24 23:10 modular-magician