nouhau icon indicating copy to clipboard operation
nouhau copied to clipboard

Published 20 hours ago • verified publisher icon gcpug

Terraform GCP Provider 1.13.0でService Enableが動かない

Open sinmetal opened this issue 6 years ago • 0 comments

Problem

Terraform GCP Provider 1.13.0 で google_project_services を利用すると、403で転ける。転けた時にAPIがDisableになるっぽいので、全体的に死ぬ。

Try Log

google_project_services.tf

resource "google_project_services" "project" {
  project = "sinmetal-terraform"

  services = ["cloudapis.googleapis.com",
    "iam.googleapis.com",
    "serviceusage.googleapis.com",
    "cloudbuild.googleapis.com",
    "cloudresourcemanager.googleapis.com",
    "storage.googleapis.com",
    "pubsub.googleapis.com",
  ]
}

terraform apply log

terraform apply
2018/06/04 20:51:24 [WARN] Invalid log level: "1". Defaulting to level: TRACE. Valid levels are: [TRACE DEBUG INFO WARN ERROR]
google_storage_bucket.sinmetal-terraform-20180327b: Refreshing state... (ID: sinmetal-terraform-20180327b)
google_storage_bucket.sinmetal-terraform-20180327a: Refreshing state... (ID: sinmetal-terraform-20180327a)
google_bigquery_dataset.log-dataset: Refreshing state... (ID: logging-sinmetal-org:organization_audit_log)
google_storage_bucket.sinmetal-terraform-20180327c: Refreshing state... (ID: sinmetal-terraform-20180327c)
google_logging_organization_sink.organization-auditlog-sink: Refreshing state... (ID: organizations/69165754818/sinks/organization-auditlog-sink)
google_logging_organization_sink.organization-gae-sink: Refreshing state... (ID: organizations/69165754818/sinks/organization-gae-sink)
google_project_iam_binding.log-writer: Refreshing state... (ID: logging-sinmetal-org/roles/bigquery.dataEditor)

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  + google_project_services.project
      id:                  <computed>
      disable_on_destroy:  "true"
      project:             "sinmetal-terraform"
      services.#:          "9"
      services.1109577435: "cloudbuild.googleapis.com"
      services.1560437671: "iam.googleapis.com"
      services.1610229196: "bigquery-json.googleapis.com"
      services.1954675454: "serviceusage.googleapis.com"
      services.2117420113: "pubsub.googleapis.com"
      services.238136042:  "cloudapis.googleapis.com"
      services.3266434626: "bigquery.googleapis.com"
      services.3644083179: "cloudresourcemanager.googleapis.com"
      services.3872232641: "storage.googleapis.com"


Plan: 1 to add, 0 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

google_project_services.project: Creating...
  disable_on_destroy:  "" => "true"
  project:             "" => "sinmetal-terraform"
  services.#:          "" => "9"
  services.1109577435: "" => "cloudbuild.googleapis.com"
  services.1560437671: "" => "iam.googleapis.com"
  services.1610229196: "" => "bigquery-json.googleapis.com"
  services.1954675454: "" => "serviceusage.googleapis.com"
  services.2117420113: "" => "pubsub.googleapis.com"
  services.238136042:  "" => "cloudapis.googleapis.com"
  services.3266434626: "" => "bigquery.googleapis.com"
  services.3644083179: "" => "cloudresourcemanager.googleapis.com"
  services.3872232641: "" => "storage.googleapis.com"
google_project_services.project: Still creating... (10s elapsed)

Error: Error applying plan:

1 error(s) occurred:

* google_project_services.project: 1 error(s) occurred:

* google_project_services.project: Error creating services: Error enabling service ["storage.googleapis.com" "iam.googleapis.com" "bigquery-json.googleapis.com" "cloudapis.googleapis.com" "bigquery.googleapis.com" "cloudbuild.googleapis.com"] for project "sinmetal-terraform": googleapi: Error 403: The caller does not have permission, forbidden

Terraform does not automatically rollback in the face of errors.
Instead, your Terraform state file has been partially updated with
any resources that successfully completed. Please address the error
above and apply again to incrementally change your infrastructure.

terraform debug log

All Log

2018-06-04T20:51:44.060+0900 [DEBUG] plugin.terraform-provider-google_v1.13.0_x4: 2018/06/04 20:51:44 [DEBUG] Got true while polling for operation operations/acf.05d81ad7-cf50-4947-834f-4da0104f443d's 'done' status
2018-06-04T20:51:44.060+0900 [DEBUG] plugin.terraform-provider-google_v1.13.0_x4: 2018/06/04 20:51:44 [DEBUG] Waiting for state to become: [success]
2018-06-04T20:51:44.060+0900 [DEBUG] plugin.terraform-provider-google_v1.13.0_x4: 2018/06/04 20:51:44 [WARN] Invalid log level: "1". Defaulting to level: TRACE. Valid levels are: [TRACE DEBUG INFO WARN ERROR]
2018-06-04T20:51:44.060+0900 [DEBUG] plugin.terraform-provider-google_v1.13.0_x4: 2018/06/04 20:51:44 [DEBUG] Google API Request Details:
2018-06-04T20:51:44.060+0900 [DEBUG] plugin.terraform-provider-google_v1.13.0_x4: ---[ REQUEST ]---------------------------------------
2018-06-04T20:51:44.060+0900 [DEBUG] plugin.terraform-provider-google_v1.13.0_x4: POST /v1beta1/projects/sinmetal-terraform/services:batchEnable?alt=json HTTP/1.1
2018-06-04T20:51:44.060+0900 [DEBUG] plugin.terraform-provider-google_v1.13.0_x4: Host: serviceusage.googleapis.com
2018-06-04T20:51:44.060+0900 [DEBUG] plugin.terraform-provider-google_v1.13.0_x4: User-Agent: google-api-go-client/0.5 Terraform/0.11.3-dev (+https://www.terraform.io)
2018-06-04T20:51:44.060+0900 [DEBUG] plugin.terraform-provider-google_v1.13.0_x4: Content-Length: 175
2018-06-04T20:51:44.060+0900 [DEBUG] plugin.terraform-provider-google_v1.13.0_x4: Content-Type: application/json
2018-06-04T20:51:44.060+0900 [DEBUG] plugin.terraform-provider-google_v1.13.0_x4: Accept-Encoding: gzip
2018-06-04T20:51:44.060+0900 [DEBUG] plugin.terraform-provider-google_v1.13.0_x4: 
2018-06-04T20:51:44.060+0900 [DEBUG] plugin.terraform-provider-google_v1.13.0_x4: 
2018-06-04T20:51:44.060+0900 [DEBUG] plugin.terraform-provider-google_v1.13.0_x4: -----------------------------------------------------
2018-06-04T20:51:44.865+0900 [DEBUG] plugin.terraform-provider-google_v1.13.0_x4: 2018/06/04 20:51:44 [WARN] Invalid log level: "1". Defaulting to level: TRACE. Valid levels are: [TRACE DEBUG INFO WARN ERROR]
2018-06-04T20:51:44.865+0900 [DEBUG] plugin.terraform-provider-google_v1.13.0_x4: 2018/06/04 20:51:44 [DEBUG] Google API Response Details:
2018-06-04T20:51:44.865+0900 [DEBUG] plugin.terraform-provider-google_v1.13.0_x4: ---[ RESPONSE ]--------------------------------------
2018-06-04T20:51:44.865+0900 [DEBUG] plugin.terraform-provider-google_v1.13.0_x4: HTTP/2.0 403 Forbidden
2018-06-04T20:51:44.865+0900 [DEBUG] plugin.terraform-provider-google_v1.13.0_x4: Alt-Svc: quic=":443"; ma=2592000; v="43,42,41,39,35"
2018-06-04T20:51:44.865+0900 [DEBUG] plugin.terraform-provider-google_v1.13.0_x4: Cache-Control: private
2018-06-04T20:51:44.865+0900 [DEBUG] plugin.terraform-provider-google_v1.13.0_x4: Content-Type: application/json; charset=UTF-8
2018-06-04T20:51:44.865+0900 [DEBUG] plugin.terraform-provider-google_v1.13.0_x4: Date: Mon, 04 Jun 2018 11:51:46 GMT
2018-06-04T20:51:44.865+0900 [DEBUG] plugin.terraform-provider-google_v1.13.0_x4: Server: ESF
2018-06-04T20:51:44.866+0900 [DEBUG] plugin.terraform-provider-google_v1.13.0_x4: Vary: Origin
2018-06-04T20:51:44.866+0900 [DEBUG] plugin.terraform-provider-google_v1.13.0_x4: Vary: X-Origin
2018-06-04T20:51:44.866+0900 [DEBUG] plugin.terraform-provider-google_v1.13.0_x4: Vary: Referer
2018-06-04T20:51:44.866+0900 [DEBUG] plugin.terraform-provider-google_v1.13.0_x4: X-Content-Type-Options: nosniff
2018-06-04T20:51:44.866+0900 [DEBUG] plugin.terraform-provider-google_v1.13.0_x4: X-Frame-Options: SAMEORIGIN
2018-06-04T20:51:44.866+0900 [DEBUG] plugin.terraform-provider-google_v1.13.0_x4: X-Xss-Protection: 1; mode=block
2018-06-04T20:51:44.866+0900 [DEBUG] plugin.terraform-provider-google_v1.13.0_x4: 
2018-06-04T20:51:44.866+0900 [DEBUG] plugin.terraform-provider-google_v1.13.0_x4: {
2018-06-04T20:51:44.866+0900 [DEBUG] plugin.terraform-provider-google_v1.13.0_x4:   "error": {
2018-06-04T20:51:44.866+0900 [DEBUG] plugin.terraform-provider-google_v1.13.0_x4:     "code": 403,
2018-06-04T20:51:44.866+0900 [DEBUG] plugin.terraform-provider-google_v1.13.0_x4:     "message": "The caller does not have permission",
2018-06-04T20:51:44.866+0900 [DEBUG] plugin.terraform-provider-google_v1.13.0_x4:     "errors": [
2018-06-04T20:51:44.866+0900 [DEBUG] plugin.terraform-provider-google_v1.13.0_x4:       {
2018-06-04T20:51:44.866+0900 [DEBUG] plugin.terraform-provider-google_v1.13.0_x4:         "message": "The caller does not have permission",
2018-06-04T20:51:44.866+0900 [DEBUG] plugin.terraform-provider-google_v1.13.0_x4:         "domain": "global",
2018-06-04T20:51:44.866+0900 [DEBUG] plugin.terraform-provider-google_v1.13.0_x4:         "reason": "forbidden"
2018-06-04T20:51:44.866+0900 [DEBUG] plugin.terraform-provider-google_v1.13.0_x4:       }
2018-06-04T20:51:44.866+0900 [DEBUG] plugin.terraform-provider-google_v1.13.0_x4:     ],
2018-06-04T20:51:44.866+0900 [DEBUG] plugin.terraform-provider-google_v1.13.0_x4:     "status": "PERMISSION_DENIED"
2018-06-04T20:51:44.866+0900 [DEBUG] plugin.terraform-provider-google_v1.13.0_x4:   }
2018-06-04T20:51:44.866+0900 [DEBUG] plugin.terraform-provider-google_v1.13.0_x4: }
2018-06-04T20:51:44.866+0900 [DEBUG] plugin.terraform-provider-google_v1.13.0_x4: 
2018-06-04T20:51:44.866+0900 [DEBUG] plugin.terraform-provider-google_v1.13.0_x4: -----------------------------------------------------

WHY

GCP Provider 1.13.0からAPIのEnable/Disableを行うAPIを、 Service Management API から Service Usage API に変更している。

https://github.com/terraform-providers/terraform-provider-google/blob/master/CHANGELOG.md#1130-may-24-2018

google_project_service/google_project_services now use the Service Usage API. Users of those resources will need to enable the API at https://console.cloud.google.com/apis/api/serviceusage.googleapis.com.

https://github.com/terraform-providers/terraform-provider-google/blob/master/CHANGELOG.md#1130-may-24-2018

その後、Service Usage API側で仕様変更か何かがあったのか、動かなくなってしまったようだ。 この問題は https://github.com/terraform-providers/terraform-provider-google/issues/1538 で議論されている。

sinmetal avatar Jun 05 '18 03:06 sinmetal