datafusion - add args + iam support

[DrFaust92]

Closes https://github.com/hashicorp/terraform-provider-google/issues/10536 Closes https://github.com/hashicorp/terraform-provider-google/issues/9065 Closes https://github.com/hashicorp/terraform-provider-google/issues/9068 Supersedes #4811

If this PR is for Terraform, I acknowledge that I have:

• [x] Searched through the issue tracker for an open issue that this either resolves or contributes to, commented on it to claim it, and written "fixes {url}" or "part of {url}" in this PR description. If there were no relevant open issues, I opened one and commented that I would like to work on it (not necessary for very small changes).
• [x] Generated Terraform, and ran make test and make lint to ensure it passes unit and linter tests.
• [x] Ensured that all new fields I added that can be set by a user appear in at least one example (for generated resources) or third_party test (for handwritten resources or update tests).
• [x] Ran relevant acceptance tests (If the acceptance tests do not yet pass or you are unable to run them, please let your reviewer know).
• [x] Read the Release Notes Guide before writing my release note below.

Release Note Template for Downstream PRs (will be copied)

datafusion: add `zone`, `display_name`, `crypto_key_config`, `event_publish_config`, and `enable_rbac` args to `google_data_fusion_instance`

datafusion: add `api_endpoint` and `p4_service_account ` attributes to `google_data_fusion_instance`

google_data_fusion_instance_iam_member

google_data_fusion_instance_iam_binding

google_data_fusion_instance_iam_policy

[modular-magician]

Hello! I am a robot who works on Magic Modules PRs.

I've detected that you're a community contributor. @scottsuarez, a repository maintainer, has been assigned to assist you and help review your changes.

:question: First time contributing? Click here for more details

---

Your assigned reviewer will help review your code by:

• Ensuring it's backwards compatible, covers common error cases, etc.
• Summarizing the change into a user-facing changelog note.
• Passes tests, either our "VCR" suite, a set of presubmit tests, or with manual test runs.

You can help make sure that review is quick by running local tests and ensuring they're passing in between each push you make to your PR's branch. Also, try to leave a comment with each push you make, as pushes generally don't generate emails.

If your reviewer doesn't get back to you within a week after your most recent change, please feel free to leave a comment on the issue asking them to take a look! In the absence of a dedicated review dashboard most maintainers manage their pending reviews through email, and those will sometimes get lost in their inbox.

---

[modular-magician]

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 8 files changed, 1101 insertions(+), 2 deletions(-)) Terraform Beta: Diff ( 8 files changed, 1101 insertions(+), 2 deletions(-)) TF Validator: Diff ( 6 files changed, 450 insertions(+), 3 deletions(-)) TF OiCS: Diff ( 13 files changed, 343 insertions(+))

[modular-magician]

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 8 files changed, 1129 insertions(+), 24 deletions(-)) Terraform Beta: Diff ( 8 files changed, 1129 insertions(+), 24 deletions(-)) TF Validator: Diff ( 6 files changed, 450 insertions(+), 3 deletions(-)) TF OiCS: Diff ( 14 files changed, 357 insertions(+), 11 deletions(-))

[modular-magician]

Tests analytics

Total tests: 2106 Passed tests 1869 Skipped tests: 226 Failed tests: 11

Action taken

Triggering VCR tests in RECORDING mode for the tests that failed during VCR. Click here to see the failed tests

TestAccDataFusionInstance_dataFusionInstanceZoneExample|TestAccDataFusionInstance_dataFusionInstanceEventExample|TestAccDataFusionInstance_dataFusionInstanceCmekExample|TestAccDataFusionInstance_dataFusionInstanceFullExample|TestAccDataFusionInstance_update|TestAccDataFusionInstanceIamPolicyGenerated|TestAccDataFusionInstanceIamMemberGenerated|TestAccDataFusionInstanceIamBindingGenerated|TestAccFirebaserulesRelease_BasicRelease|TestAccPrivatecaCertificateAuthority_privatecaCertificateAuthoritySubordinateExample|TestAccPrivatecaCertificateAuthority_subordinateCaActivatedByFirstPartyIssuerOnCreation

[modular-magician]

Tests passed during RECORDING mode: TestAccDataFusionInstanceIamBindingGenerated[view] TestAccDataFusionInstance_dataFusionInstanceZoneExample[view] TestAccFirebaserulesRelease_BasicRelease[view] TestAccDataFusionInstance_dataFusionInstanceEventExample[view] TestAccDataFusionInstanceIamPolicyGenerated[view] TestAccDataFusionInstanceIamMemberGenerated[view]

Tests failed during RECORDING mode: TestAccPrivatecaCertificateAuthority_subordinateCaActivatedByFirstPartyIssuerOnCreation[view] TestAccPrivatecaCertificateAuthority_privatecaCertificateAuthoritySubordinateExample[view] TestAccDataFusionInstance_update[view] TestAccDataFusionInstance_dataFusionInstanceFullExample[view] TestAccDataFusionInstance_dataFusionInstanceCmekExample[view]

Please fix these to complete your PR View the build log or the debug log for each test

[modular-magician]

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 8 files changed, 1181 insertions(+), 26 deletions(-)) Terraform Beta: Diff ( 8 files changed, 1181 insertions(+), 26 deletions(-)) TF Validator: Diff ( 6 files changed, 450 insertions(+), 3 deletions(-)) TF OiCS: Diff ( 14 files changed, 383 insertions(+), 12 deletions(-))

[modular-magician]

Tests analytics

Total tests: 2114 Passed tests 1880 Skipped tests: 226 Failed tests: 8

Action taken

Triggering VCR tests in RECORDING mode for the tests that failed during VCR. Click here to see the failed tests

TestAccPrivatecaCertificateAuthority_subordinateCaActivatedByFirstPartyIssuerOnCreation|TestAccPrivatecaCertificateAuthority_privatecaCertificateAuthoritySubordinateExample|TestAccDataSourceGoogleServiceAccountIdToken_basic|TestAccFirebaserulesRelease_BasicRelease|TestAccDataFusionInstance_dataFusionInstanceCmekExample|TestAccDataFusionInstance_dataFusionInstanceFullExample|TestAccDataFusionInstance_update|TestAccComputeInstance_soleTenantNodeAffinities

[modular-magician]

Tests passed during RECORDING mode: TestAccDataSourceGoogleServiceAccountIdToken_basic[view] TestAccFirebaserulesRelease_BasicRelease[view] TestAccDataFusionInstance_dataFusionInstanceCmekExample[view]

Tests failed during RECORDING mode: TestAccPrivatecaCertificateAuthority_subordinateCaActivatedByFirstPartyIssuerOnCreation[view] TestAccPrivatecaCertificateAuthority_privatecaCertificateAuthoritySubordinateExample[view] TestAccDataFusionInstance_update[view] TestAccDataFusionInstance_dataFusionInstanceFullExample[view] TestAccComputeInstance_soleTenantNodeAffinities[view]

Please fix these to complete your PR View the build log or the debug log for each test

[modular-magician]

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 8 files changed, 1182 insertions(+), 26 deletions(-)) Terraform Beta: Diff ( 8 files changed, 1182 insertions(+), 26 deletions(-)) TF Validator: Diff ( 6 files changed, 450 insertions(+), 3 deletions(-)) TF OiCS: Diff ( 14 files changed, 383 insertions(+), 12 deletions(-))

[modular-magician]

Tests analytics

Total tests: 2114 Passed tests 1882 Skipped tests: 226 Failed tests: 6

Action taken

Triggering VCR tests in RECORDING mode for the tests that failed during VCR. Click here to see the failed tests

TestAccDataFusionInstance_dataFusionInstanceFullExample|TestAccComputeInstance_soleTenantNodeAffinities|TestAccPrivatecaCertificateAuthority_privatecaCertificateAuthoritySubordinateExample|TestAccFirebaserulesRelease_BasicRelease|TestAccPrivatecaCertificateAuthority_subordinateCaActivatedByFirstPartyIssuerOnCreation|TestAccSqlUser_mysqlDisabled

[modular-magician]

Tests passed during RECORDING mode: TestAccSqlUser_mysqlDisabled[view] TestAccFirebaserulesRelease_BasicRelease[view]

Tests failed during RECORDING mode: TestAccComputeInstance_soleTenantNodeAffinities[view] TestAccPrivatecaCertificateAuthority_subordinateCaActivatedByFirstPartyIssuerOnCreation[view] TestAccPrivatecaCertificateAuthority_privatecaCertificateAuthoritySubordinateExample[view] TestAccDataFusionInstance_dataFusionInstanceFullExample[view]

Please fix these to complete your PR View the build log or the debug log for each test

[DrFaust92]

unsure whats the TestAccDataFusionInstance_dataFusionInstanceFullExample failure as locally it passes and i cant access logs

[ScottSuarez]

looks to be a transient error

 Error: Error waiting to create Instance: Error waiting for Creating Instance: Error code 13, message: an internal error has occurred

[ScottSuarez]

/gcbrun

[modular-magician]

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 8 files changed, 1182 insertions(+), 26 deletions(-)) Terraform Beta: Diff ( 8 files changed, 1182 insertions(+), 26 deletions(-)) TF Validator: Diff ( 6 files changed, 450 insertions(+), 3 deletions(-)) TF OiCS: Diff ( 14 files changed, 383 insertions(+), 12 deletions(-))

[modular-magician]

Tests analytics

Total tests: 2117 Passed tests 1886 Skipped tests: 226 Failed tests: 5

Action taken

Triggering VCR tests in RECORDING mode for the tests that failed during VCR. Click here to see the failed tests

TestAccDataFusionInstance_dataFusionInstanceFullExample|TestAccComputeInstance_soleTenantNodeAffinities|TestAccPrivatecaCertificateAuthority_subordinateCaActivatedByFirstPartyIssuerOnCreation|TestAccSqlUser_mysqlDisabled|TestAccPrivatecaCertificateAuthority_privatecaCertificateAuthoritySubordinateExample

[modular-magician]

Tests passed during RECORDING mode: TestAccSqlUser_mysqlDisabled[view]

Tests failed during RECORDING mode: TestAccComputeInstance_soleTenantNodeAffinities[view] TestAccPrivatecaCertificateAuthority_subordinateCaActivatedByFirstPartyIssuerOnCreation[view] TestAccPrivatecaCertificateAuthority_privatecaCertificateAuthoritySubordinateExample[view] TestAccDataFusionInstance_dataFusionInstanceFullExample[view]

Please fix these to complete your PR View the build log or the debug log for each test

[ScottSuarez]

hmm, I'm not sure to be honest.

        Error: Error waiting to create Instance: Error waiting for Creating Instance: Error code 13, message: an internal error has occurred
        
          with google_data_fusion_instance.extended_instance,
          on terraform_plugin_test.tf line 2, in resource "google_data_fusion_instance" "extended_instance":
           2: resource "google_data_fusion_instance" "extended_instance" {

[modular-magician]

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 7 files changed, 1176 insertions(+), 26 deletions(-)) Terraform Beta: Diff ( 7 files changed, 1176 insertions(+), 26 deletions(-)) TF Validator: Diff ( 6 files changed, 450 insertions(+), 3 deletions(-)) TF OiCS: Diff ( 14 files changed, 383 insertions(+), 12 deletions(-))

[modular-magician]

Tests analytics

Total tests: 2120 Passed tests 1889 Skipped tests: 226 Failed tests: 5

Action taken

Triggering VCR tests in RECORDING mode for the tests that failed during VCR. Click here to see the failed tests

TestAccComputeInstance_soleTenantNodeAffinities|TestAccFirebaserulesRelease_BasicRelease|TestAccDataFusionInstance_dataFusionInstanceFullExample|TestAccPrivatecaCertificateAuthority_subordinateCaActivatedByFirstPartyIssuerOnCreation|TestAccPrivatecaCertificateAuthority_privatecaCertificateAuthoritySubordinateExample

[modular-magician]

Tests passed during RECORDING mode: TestAccPrivatecaCertificateAuthority_privatecaCertificateAuthoritySubordinateExample[view] TestAccFirebaserulesRelease_BasicRelease[view]

Tests failed during RECORDING mode: TestAccComputeInstance_soleTenantNodeAffinities[view] TestAccPrivatecaCertificateAuthority_subordinateCaActivatedByFirstPartyIssuerOnCreation[view] TestAccDataFusionInstance_dataFusionInstanceFullExample[view]

Please fix these to complete your PR View the build log or the debug log for each test

[ScottSuarez]

I've filled a bug with the owning team.. We'll see what they say on why this might be failing

[modular-magician]

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 7 files changed, 1176 insertions(+), 26 deletions(-)) Terraform Beta: Diff ( 7 files changed, 1176 insertions(+), 26 deletions(-)) TF Validator: Diff ( 6 files changed, 450 insertions(+), 3 deletions(-)) TF OiCS: Diff ( 14 files changed, 383 insertions(+), 12 deletions(-))

[modular-magician]

Tests analytics

Total tests: 2131 Passed tests 1892 Skipped tests: 227 Failed tests: 12

Action taken

Triggering VCR tests in RECORDING mode for the tests that failed during VCR. Click here to see the failed tests

TestAccFirebaserulesRelease_BasicRelease|TestAccDataFusionInstance_dataFusionInstanceFullExample|TestAccComputeInstance_soleTenantNodeAffinities|TestAccComputeInstance_networkPerformanceConfig|TestAccComputeGlobalForwardingRule_internalLoadBalancing|TestAccCloudfunctions2function_cloudfunctions2BasicGcsExample|TestAccCloudRunService_cloudRunServiceStaticOutboundExample|TestAccCloudfunctions2function_cloudfunctions2BasicAuditlogsExample|TestAccCloudFunctions2Function_fullUpdate|TestAccCGCSnippet_eventarcWorkflowsExample|TestAccPrivatecaCertificateAuthority_privatecaCertificateAuthoritySubordinateExample|TestAccSqlUser_mysqlDisabled

[modular-magician]

Tests passed during RECORDING mode: TestAccFirebaserulesRelease_BasicRelease[view] TestAccCGCSnippet_eventarcWorkflowsExample[view] TestAccPrivatecaCertificateAuthority_privatecaCertificateAuthoritySubordinateExample[view] TestAccCloudfunctions2function_cloudfunctions2BasicGcsExample[view] TestAccCloudfunctions2function_cloudfunctions2BasicAuditlogsExample[view] TestAccCloudFunctions2Function_fullUpdate[view] TestAccSqlUser_mysqlDisabled[view]

Tests failed during RECORDING mode: TestAccComputeInstance_networkPerformanceConfig[view] TestAccComputeGlobalForwardingRule_internalLoadBalancing[view] TestAccComputeInstance_soleTenantNodeAffinities[view] TestAccCloudRunService_cloudRunServiceStaticOutboundExample[view] TestAccDataFusionInstance_dataFusionInstanceFullExample[view]

Please fix these to complete your PR View the build log or the debug log for each test

[ScottSuarez]

Looks like these tests fail due to the compute network allocations. Can we rely on shared resources or align the implementations with existing tests?

[modular-magician]

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 7 files changed, 1199 insertions(+), 26 deletions(-)) Terraform Beta: Diff ( 7 files changed, 1199 insertions(+), 26 deletions(-)) TF Validator: Diff ( 6 files changed, 451 insertions(+), 3 deletions(-)) TF OiCS: Diff ( 14 files changed, 384 insertions(+), 12 deletions(-))

[modular-magician]

Tests analytics

Total tests: 2145 Passed tests 1900 Skipped tests: 228 Failed tests: 17

Action taken

Triggering VCR tests in RECORDING mode for the tests that failed during VCR. Click here to see the failed tests

TestAccDataFusionInstanceIamBindingGenerated|TestAccDataFusionInstanceIamMemberGenerated|TestAccDataFusionInstanceIamPolicyGenerated|TestAccFirebaserulesRelease_BasicRelease|TestAccDataFusionInstance_dataFusionInstanceFullExample|TestAccComputeInstance_soleTenantNodeAffinities|TestAccComputeInstance_networkPerformanceConfig|TestAccComputeGlobalForwardingRule_internalLoadBalancing|TestAccCloudfunctions2function_cloudfunctions2BasicGcsExample|TestAccCloudfunctions2function_cloudfunctions2BasicAuditlogsExample|TestAccBigtableAppProfile_bigtableAppProfileMulticlusterExample|TestAccBigtableAppProfile_bigtableAppProfileAnyclusterExample|TestAccCloudFunctions2Function_fullUpdate|TestAccCloudRunService_cloudRunServiceStaticOutboundExample|TestAccPrivatecaCertificateAuthority_privatecaCertificateAuthoritySubordinateExample|TestAccSqlDatabaseInstance_withPrivateNetwork_withAllocatedIpRange|TestAccSqlDatabaseInstance_SqlServerAuditConfig

[modular-magician]

Tests passed during RECORDING mode: TestAccDataFusionInstanceIamBindingGenerated[view] TestAccDataFusionInstanceIamMemberGenerated[view] TestAccDataFusionInstanceIamPolicyGenerated[view] TestAccFirebaserulesRelease_BasicRelease[view] TestAccCloudfunctions2function_cloudfunctions2BasicAuditlogsExample[view] TestAccBigtableAppProfile_bigtableAppProfileMulticlusterExample[view] TestAccBigtableAppProfile_bigtableAppProfileAnyclusterExample[view] TestAccCloudFunctions2Function_fullUpdate[view] TestAccPrivatecaCertificateAuthority_privatecaCertificateAuthoritySubordinateExample[view] TestAccSqlDatabaseInstance_withPrivateNetwork_withAllocatedIpRangeReplica[view]

Tests failed during RECORDING mode: TestAccDataFusionInstance_dataFusionInstanceFullExample[view] TestAccComputeInstance_soleTenantNodeAffinities[view] TestAccComputeInstance_networkPerformanceConfig[view] TestAccComputeGlobalForwardingRule_internalLoadBalancing[view] TestAccCloudfunctions2function_cloudfunctions2BasicGcsExample[view] TestAccCloudRunService_cloudRunServiceStaticOutboundExample[view] TestAccSqlDatabaseInstance_withPrivateNetwork_withAllocatedIpRange[view] TestAccSqlDatabaseInstance_withPrivateNetwork_withAllocatedIpRangeClone[view] TestAccSqlDatabaseInstance_SqlServerAuditConfig[view]

Please fix these to complete your PR View the build log or the debug log for each test

[imrannayer]

Is there an ETA for this release?