google_compute_instance not respecting shared-VPC granular subnetwork-permissions

[pritho]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request.
• Please do not leave +1 or me too comments, they generate extra noise for issue followers and do not help prioritize the request.
• If you are interested in working on this issue or have submitted a pull request, please leave a comment.
• If an issue is assigned to the modular-magician user, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If an issue is assigned to a user, that user is claiming responsibility for the issue. If an issue is assigned to hashibot, a community member has claimed the issue already.

Terraform Version

0.15.5

Affected Resource(s)

• google_compute_instance
• google_compute_network
• google_compute_subnetwork

Terraform Configuration Files

resource "google_compute_instance" "instance" {
  name     = local.hostname
  machine_type = var.machine_type
  zone         = var.gcp_zone
  boot_disk {
    source = google_compute_disk.rootvol.name
  }
  network_interface {
    subnetwork         = var.subnetwork
    subnetwork_project = var.subnetwork_project
  }
}

Expected Behavior

Instance should be able to be created

Actual Behavior

Error: Error updating network interface: googleapi: Error 403: Required 'compute.networks.use' permission for 'projects/HOST/global/networks/X', forbidden

Steps to Reproduce

1. Create a shared network X in Project HOST
2. Attach a project SERVICE
3. Create a service account CHILD in project SERVICE
4. Create subnetwork Y in network X in HOST
5. Share subnetwork Y with user CHILD from project HOST with IAM subnet-policy and role "roles/compute.networkUser"
6. Create an instance in project SERVICE using service account CHILD in shared subnetwork Y

Important Factoids

When granting the global permission "roles/compute.networkUser" via IAM in project HOST to service account CHILD, the setup works, but this on the other hand gives direct permissions to use all subnetworks.

b/308756106

[karolgorc]

Couldn't recreate the issue. None of the permissions are provided in terraform code so i'd suggest checking out if they are pointing to the right projects

The compute.networkUser permission worked

resource "google_project_iam_custom_role" "custom_role_child" {
  role_id = "test"
  title   = "test-child"
  permissions = [
    "compute.instances.create",
    "compute.disks.create",
    "compute.subnetworks.use",
  ]
  project = var.project_child
}

resource "google_project_iam_member" "service_account_role_host" {
  for_each = toset(["roles/compute.networkUser", "roles/viewer"])
  project  = var.project_host
  role     = each.key
  member   = "serviceAccount:${google_service_account.service_account.email}"
}

resource "google_project_iam_member" "service_account_role_child" {
  for_each = toset(["${google_project_iam_custom_role.custom_role_child.id}", "roles/viewer"])
  project  = var.project_child
  role     = each.key
  member   = "serviceAccount:${google_service_account.service_account.email}"
}

And here is a more granular version that also works

resource "google_project_iam_custom_role" "custom_role_host" {
  role_id = "test"
  title   = "test-host"
  permissions = [
    "compute.networks.use",
    "compute.subnetworks.use",
  ]
  project = var.project_host
}

resource "google_project_iam_custom_role" "custom_role_child" {
  role_id = "test"
  title   = "test-child"
  permissions = [
    "compute.instances.create",
    "compute.disks.create",
    "compute.subnetworks.use",
  ]
  project = var.project_child
}

resource "google_project_iam_member" "service_account_role_host" {
  for_each = toset(["${google_project_iam_custom_role.custom_role_host.id}", "roles/viewer"])
  project  = var.project_host
  role     = each.key
  member   = "serviceAccount:${google_service_account.service_account.email}"
}

resource "google_project_iam_member" "service_account_role_child" {
  for_each = toset(["${google_project_iam_custom_role.custom_role_child.id}", "roles/viewer"])
  project  = var.project_child
  role     = each.key
  member   = "serviceAccount:${google_service_account.service_account.email}"
}

@edwardmedia I think this issue can be closed

[andrewkaczynski]

I have the same issue! The subnet was shared with me from HOST project network, while I have permissions only to my project. When using a service account with OWNER role on my project, I got an error about lack of permissions to read subnet from HOST project:

Error 403: Required 'compute.subnetworks.use'

provider:

  required_providers {
    google = {
      source  = "hashicorp/google"
      version = "6.21.0"
    }

[rileykarson]

I confirmed this worked with subnet-level permissions explicitly with the below repro (infra/main.tf with a highly permissioned account, and application/main.tf with the SA created in infra/main.tf), and project-level permissions were confirmed above. I believe you need specific permissions against the subnetwork or host project with the principal performing the attachment- typically roles/compute.networkUser as I used here.

infra/main.tf

resource "google_project" "host" {
  name = "test-host-project-4365658483"
  project_id = "test-host-project-4365658483"
  org_id = "<org>"
  billing_account = "<ba>"
}

resource "google_project_service" "host_compute" {
  project = google_project.host.project_id
  service = "compute.googleapis.com"
  disable_on_destroy = false
}

resource "google_compute_shared_vpc_host_project" "host" {
  project = google_project.host.project_id
  depends_on = [google_project_service.host_compute]
}


resource "google_project" "service" {
  name       = "test-service-project-1243325"
  project_id = "test-service-project-1243325"
  org_id = "<org>"
  billing_account = "<ba>"
}

resource "google_project_service" "service_compute" {
  project = google_project.service.project_id
  service = "compute.googleapis.com"
  disable_on_destroy = false
}

resource "google_service_account" "service_account" {
  account_id   = "service-account-id"
  display_name = "Service Account"
  project = google_project.service.project_id
}

# Use a more granular permission!
resource "google_project_iam_member" "sa_editor" {
  project = google_project.service.project_id
  role = "roles/editor"
  member = google_service_account.service_account.member
}

resource "google_compute_shared_vpc_service_project" "service" {
  host_project    = google_compute_shared_vpc_host_project.host.project
  service_project = google_project.service.project_id
  depends_on = [google_project_service.host_compute, google_project_service.service_compute]
}

resource "google_compute_subnetwork_iam_member" "member" {
  project = google_project.host.project_id
  region = "us-central1"
  subnetwork = "default"
  role = "roles/compute.networkUser"
  member = google_service_account.service_account.member

  depends_on = [google_project_service.host_compute]
}

# not strictly necessary for the repro, but makes it easy to impersonate
resource "google_service_account_iam_member" "add-impersonation-permissions" {
  service_account_id = google_service_account.service_account.id

  member             = "user:<email>"
  role               = "roles/iam.serviceAccountTokenCreator"
}

application/main.tf

resource "google_compute_instance" "instance" {
  name         = "my-machine"
  zone         = "us-central1-f"
  project      = "test-service-project-1243325"
  machine_type = "e2-medium"

  boot_disk {
    initialize_params {
      image = "debian-cloud/debian-11"
    }
  }

  network_interface {
    subnetwork         = "default"
    subnetwork_project = "test-host-project-4365658483"
  }
}

terraform apply

$ GOOGLE_IMPERSONATE_SERVICE_ACCOUNT=service-account-id@test-service-project-1243325.iam.gserviceaccount.com terraform apply

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # google_compute_instance.instance will be created
  + resource "google_compute_instance" "instance" {
      + can_ip_forward       = false
      + cpu_platform         = (known after apply)
      + creation_timestamp   = (known after apply)
      + current_status       = (known after apply)
      + deletion_protection  = false
      + effective_labels     = {
          + "goog-terraform-provisioned" = "true"
        }
      + id                   = (known after apply)
      + instance_id          = (known after apply)
      + label_fingerprint    = (known after apply)
      + machine_type         = "e2-medium"
      + metadata_fingerprint = (known after apply)
      + min_cpu_platform     = (known after apply)
      + name                 = "my-machine"
      + project              = "test-service-project-1243325"
      + self_link            = (known after apply)
      + tags_fingerprint     = (known after apply)
      + terraform_labels     = {
          + "goog-terraform-provisioned" = "true"
        }
      + zone                 = "us-central1-f"

      + boot_disk {
          + auto_delete                = true
          + device_name                = (known after apply)
          + disk_encryption_key_sha256 = (known after apply)
          + guest_os_features          = (known after apply)
          + kms_key_self_link          = (known after apply)
          + mode                       = "READ_WRITE"
          + source                     = (known after apply)

          + initialize_params {
              + architecture           = (known after apply)
              + image                  = "debian-cloud/debian-11"
              + labels                 = (known after apply)
              + provisioned_iops       = (known after apply)
              + provisioned_throughput = (known after apply)
              + resource_policies      = (known after apply)
              + size                   = (known after apply)
              + snapshot               = (known after apply)
              + type                   = (known after apply)
            }
        }

      + network_interface {
          + internal_ipv6_prefix_length = (known after apply)
          + ipv6_access_type            = (known after apply)
          + ipv6_address                = (known after apply)
          + name                        = (known after apply)
          + network                     = (known after apply)
          + network_attachment          = (known after apply)
          + network_ip                  = (known after apply)
          + stack_type                  = (known after apply)
          + subnetwork                  = "default"
          + subnetwork_project          = "test-host-project-4365658483"
        }
    }

Plan: 1 to add, 0 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

google_compute_instance.instance: Creating...
google_compute_instance.instance: Still creating... [10s elapsed]
google_compute_instance.instance: Still creating... [20s elapsed]
google_compute_instance.instance: Creation complete after 29s [id=projects/test-service-project-1243325/zones/us-central1-f/instances/my-machine]

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

From https://console.cloud.google.com/compute/instancesDetail/zones/us-central1-f/instances/my-machine?inv=1&invt=Abts7A&project=test-service-project-1243325

[github-actions[bot]]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.