tf apply for azure hangs on waiting for module.jx-boot

Open royb-sw opened this issue 4 years ago • 0 comments

I'm trying to follow the quick setup guide for Azure (https://jenkins-x.io/v3/admin/platforms/azure/)

I followed the entire process (including service-principle creation) and chose as a cluster repo the Azure key Vault repo For some reason, during terraform plan/apply stage, I keep getting the TRACE/ERROR logs as follow

2022-01-25T15:46:49.205+0200 [TRACE] dag/walk: vertex "module.cluster.module.jx-boot (close)" is waiting for "module.cluster.module.jx-boot.module.jx-health (close)"
2022-01-25T15:46:53.075+0200 [TRACE] dag/walk: vertex "module.cluster (close)" is waiting for "module.cluster.module.jx-boot (close)"
2022-01-25T15:46:53.085+0200 [TRACE] dag/walk: vertex "root" is waiting for "module.cluster.provider[\"registry.terraform.io/hashicorp/helm\"] (close)"
2022-01-25T15:46:54.206+0200 [TRACE] dag/walk: vertex "module.cluster.module.jx-boot (close)" is waiting for "module.cluster.module.jx-boot.module.jx-health (close)"
2022-01-25T15:46:54.206+0200 [TRACE] dag/walk: vertex "module.cluster.provider[\"registry.terraform.io/hashicorp/helm\"] (close)" is waiting for "module.cluster.module.jx-boot.module.jx-health.helm_release.health-checks-install (expand)"
2022-01-25T15:46:54.206+0200 [TRACE] dag/walk: vertex "module.cluster.module.jx-boot.module.jx-health (close)" is waiting for "module.cluster.module.jx-boot.module.jx-health.helm_release.cluster-checks (expand)"

Any hints on what can be the root cause for this issue? the script is running for more than 15 minutes.

adding full logs with INFO level

2022-01-26T08:44:31.442+0200 [INFO]  Terraform version: 1.1.3
2022-01-26T08:44:31.442+0200 [INFO]  Go runtime version: go1.17.2
2022-01-26T08:44:31.443+0200 [INFO]  CLI args: []string{"terraform", "plan"}
2022-01-26T08:44:31.443+0200 [INFO]  CLI command args: []string{"plan"}
2022-01-26T08:44:31.962+0200 [INFO]  backend/local: starting Plan operation
2022-01-26T08:44:32.346+0200 [INFO]  ReferenceTransformer: reference not found: "path.module"
2022-01-26T08:44:32.349+0200 [INFO]  ReferenceTransformer: reference not found: "each.key"
2022-01-26T08:44:32.349+0200 [INFO]  ReferenceTransformer: reference not found: "each.value"
2022-01-26T08:44:32.760+0200 [INFO]  backend/local: plan calling Plan
2022-01-26T08:44:32.771+0200 [INFO]  ReferenceTransformer: reference not found: "path.module"
2022-01-26T08:44:32.771+0200 [INFO]  ReferenceTransformer: reference not found: "each.value"
2022-01-26T08:44:32.771+0200 [INFO]  ReferenceTransformer: reference not found: "each.key"
2022-01-26T08:44:32.890+0200 [WARN]  Provider "registry.terraform.io/hashicorp/random" produced an invalid plan for module.cluster.random_pet.name, but we are tolerating it because it is using the legacy plugin SDK.
    The following problems may be the cause of any confusing errors from downstream operations:
      - .length: planned value cty.NumberIntVal(2) for a non-computed attribute
      - .separator: planned value cty.StringVal("-") for a non-computed attribute
2022-01-26T08:44:32.982+0200 [WARN]  ValidateProviderConfig from "module.cluster.provider[\"registry.terraform.io/hashicorp/azurerm\"]" changed the config value, but that value is unused
2022-01-26T08:44:41.233+0200 [INFO]  ReferenceTransformer: reference not found: "var.location"
2022-01-26T08:44:41.233+0200 [INFO]  ReferenceTransformer: reference not found: "local.network_resource_group_name"
2022-01-26T08:44:41.233+0200 [INFO]  ReferenceTransformer: reference not found: "local.cluster_resource_group_name"
2022-01-26T08:44:41.233+0200 [INFO]  ReferenceTransformer: reference not found: "var.location"
2022-01-26T08:44:41.234+0200 [INFO]  ReferenceTransformer: reference not found: "var.apex_resource_group_name"
2022-01-26T08:44:41.234+0200 [INFO]  ReferenceTransformer: reference not found: "var.enabled"
2022-01-26T08:44:41.234+0200 [INFO]  ReferenceTransformer: reference not found: "local.resource_group_name"
2022-01-26T08:44:41.234+0200 [INFO]  ReferenceTransformer: reference not found: "var.location"
2022-01-26T08:44:41.235+0200 [ERROR] vertex "module.cluster.module.dns.data.azurerm_resource_group.apex_resource_group" error: "name" cannot be blank
2022-01-26T08:44:41.235+0200 [ERROR] vertex "module.cluster.module.dns.data.azurerm_resource_group.apex_resource_group" error: "name" cannot be blank
2022-01-26T08:44:41.235+0200 [ERROR] vertex "module.cluster.module.dns.data.azurerm_resource_group.apex_resource_group (expand)" error: "name" cannot be blank
2022-01-26T08:44:41.235+0200 [INFO]  ReferenceTransformer: reference not found: "var.use_existing_acr_name"
2022-01-26T08:44:41.235+0200 [INFO]  ReferenceTransformer: reference not found: "local.resource_group_name"
2022-01-26T08:44:41.235+0200 [INFO]  ReferenceTransformer: reference not found: "var.location"
2022-01-26T08:44:41.235+0200 [INFO]  ReferenceTransformer: reference not found: "local.resource_group_name"
2022-01-26T08:44:41.235+0200 [INFO]  ReferenceTransformer: reference not found: "var.location"
2022-01-26T08:44:41.236+0200 [INFO]  ReferenceTransformer: reference not found: "var.resource_group"
2022-01-26T08:44:41.236+0200 [INFO]  ReferenceTransformer: reference not found: "var.location"
2022-01-26T08:44:41.236+0200 [INFO]  ReferenceTransformer: reference not found: "var.network_name"
2022-01-26T08:44:41.236+0200 [INFO]  ReferenceTransformer: reference not found: "var.vnet_cidr"
2022-01-26T08:44:41.236+0200 [INFO]  ReferenceTransformer: reference not found: "var.use_existing_acr_name"
2022-01-26T08:44:41.236+0200 [INFO]  ReferenceTransformer: reference not found: "var.location"
2022-01-26T08:44:41.236+0200 [INFO]  ReferenceTransformer: reference not found: "local.container_registry_name"
2022-01-26T08:44:41.237+0200 [INFO]  ReferenceTransformer: reference not found: "var.location"
2022-01-26T08:44:41.237+0200 [INFO]  ReferenceTransformer: reference not found: "local.account_name"
2022-01-26T08:44:41.237+0200 [WARN]  Provider "registry.terraform.io/hashicorp/azurerm" produced an invalid plan for module.cluster.module.cluster.module.vnet.azurerm_virtual_network.cluster, but we are tolerating it because it is using the legacy plugin SDK.
    The following problems may be the cause of any confusing errors from downstream operations:
      - .vm_protection_enabled: planned value cty.False for a non-computed attribute
2022-01-26T08:44:41.237+0200 [INFO]  ReferenceTransformer: reference not found: "var.subnet_name"
2022-01-26T08:44:41.237+0200 [INFO]  ReferenceTransformer: reference not found: "var.subnet_cidr"
2022-01-26T08:44:41.237+0200 [INFO]  ReferenceTransformer: reference not found: "var.resource_group"
2022-01-26T08:44:41.238+0200 [WARN]  Provider "registry.terraform.io/hashicorp/azurerm" produced an invalid plan for module.cluster.module.cluster.module.vnet.azurerm_subnet.cluster_subnet, but we are tolerating it because it is using the legacy plugin SDK.
    The following problems may be the cause of any confusing errors from downstream operations:
      - .enforce_private_link_service_network_policies: planned value cty.False for a non-computed attribute
      - .enforce_private_link_endpoint_network_policies: planned value cty.False for a non-computed attribute
2022-01-26T08:44:41.239+0200 [WARN]  Provider "registry.terraform.io/hashicorp/azurerm" produced an invalid plan for module.cluster.module.registry.azurerm_container_registry.acr[0], but we are tolerating it because it is using the legacy plugin SDK.
    The following problems may be the cause of any confusing errors from downstream operations:
      - .network_rule_bypass_option: planned value cty.StringVal("AzureServices") for a non-computed attribute
      - .zone_redundancy_enabled: planned value cty.False for a non-computed attribute
      - .public_network_access_enabled: planned value cty.True for a non-computed attribute
      - .identity: attribute representing nested block must not be unknown itself; set nested attribute values to unknown instead
2022-01-26T08:44:41.240+0200 [INFO]  ReferenceTransformer: reference not found: "var.dns_prefix"
2022-01-26T08:44:41.240+0200 [INFO]  ReferenceTransformer: reference not found: "var.resource_group_name"
2022-01-26T08:44:41.240+0200 [INFO]  ReferenceTransformer: reference not found: "var.node_resource_group_name"
2022-01-26T08:44:41.240+0200 [INFO]  ReferenceTransformer: reference not found: "var.cluster_version"
2022-01-26T08:44:41.240+0200 [INFO]  ReferenceTransformer: reference not found: "var.location"
2022-01-26T08:44:41.240+0200 [INFO]  ReferenceTransformer: reference not found: "var.cluster_name"
2022-01-26T08:44:41.240+0200 [INFO]  ReferenceTransformer: reference not found: "var.min_node_count"
2022-01-26T08:44:41.240+0200 [INFO]  ReferenceTransformer: reference not found: "var.cluster_version"
2022-01-26T08:44:41.240+0200 [INFO]  ReferenceTransformer: reference not found: "var.max_node_count"
2022-01-26T08:44:41.240+0200 [INFO]  ReferenceTransformer: reference not found: "var.vnet_subnet_id"
2022-01-26T08:44:41.240+0200 [INFO]  ReferenceTransformer: reference not found: "var.node_size"
2022-01-26T08:44:41.240+0200 [INFO]  ReferenceTransformer: reference not found: "var.node_count"
2022-01-26T08:44:41.240+0200 [INFO]  ReferenceTransformer: reference not found: "var.max_node_count"
2022-01-26T08:44:41.240+0200 [INFO]  ReferenceTransformer: reference not found: "var.cluster_network_model"
2022-01-26T08:44:41.240+0200 [INFO]  ReferenceTransformer: reference not found: "var.enable_log_analytics"
2022-01-26T08:44:41.240+0200 [INFO]  ReferenceTransformer: reference not found: "var.enable_log_analytics"
2022-01-26T08:44:41.240+0200 [INFO]  ReferenceTransformer: reference not found: "var.enable_log_analytics"
2022-01-26T08:44:41.242+0200 [WARN]  Provider "registry.terraform.io/hashicorp/azurerm" produced an invalid plan for module.cluster.module.storage.azurerm_storage_account.storage, but we are tolerating it because it is using the legacy plugin SDK.
The following problems may be the cause of any confusing errors from downstream operations:
      - .table_encryption_key_type: planned value cty.StringVal("Service") for a non-computed attribute
      - .allow_blob_public_access: planned value cty.False for a non-computed attribute
      - .infrastructure_encryption_enabled: planned value cty.False for a non-computed attribute
      - .nfsv3_enabled: planned value cty.False for a non-computed attribute
      - .queue_encryption_key_type: planned value cty.StringVal("Service") for a non-computed attribute
      - .min_tls_version: planned value cty.StringVal("TLS1_0") for a non-computed attribute
      - .shared_access_key_enabled: planned value cty.True for a non-computed attribute
      - .enable_https_traffic_only: planned value cty.True for a non-computed attribute
      - .blob_properties: attribute representing nested block must not be unknown itself; set nested attribute values to unknown instead
      - .queue_properties: attribute representing nested block must not be unknown itself; set nested attribute values to unknown instead
      - .routing: attribute representing nested block must not be unknown itself; set nested attribute values to unknown instead
      - .identity: attribute representing nested block must not be unknown itself; set nested attribute values to unknown instead
      - .network_rules: attribute representing nested block must not be unknown itself; set nested attribute values to unknown instead
      - .share_properties: attribute representing nested block must not be unknown itself; set nested attribute values to unknown instead
2022-01-26T08:44:41.245+0200 [WARN]  Provider "registry.terraform.io/hashicorp/azurerm" produced an invalid plan for module.cluster.module.storage.azurerm_storage_container.logs, but we are tolerating it because it is using the legacy plugin SDK.
    The following problems may be the cause of any confusing errors from downstream operations:
      - .container_access_type: planned value cty.StringVal("private") for a non-computed attribute
2022-01-26T08:44:41.250+0200 [WARN]  Provider "registry.terraform.io/hashicorp/azurerm" produced an invalid plan for module.cluster.module.cluster.module.cluster.azurerm_kubernetes_cluster.aks, but we are tolerating it because it is using the legacy plugin SDK.
    The following problems may be the cause of any confusing errors from downstream operations:
      - .private_cluster_public_fqdn_enabled: planned value cty.False for a non-computed attribute
      - .public_network_access_enabled: planned value cty.True for a non-computed attribute
      - .sku_tier: planned value cty.StringVal("Free") for a non-computed attribute
      - .auto_scaler_profile: attribute representing nested block must not be unknown itself; set nested attribute values to unknown instead
      - .default_node_pool[0].os_disk_type: planned value cty.StringVal("Managed") for a non-computed attribute
      - .default_node_pool[0].type: planned value cty.StringVal("VirtualMachineScaleSets") for a non-computed attribute
      - .default_node_pool[0].ultra_ssd_enabled: planned value cty.False for a non-computed attribute
      - .kubelet_identity: attribute representing nested block must not be unknown itself; set nested attribute values to unknown instead
      - .windows_profile: attribute representing nested block must not be unknown itself; set nested attribute values to unknown instead
      - .network_profile[0].load_balancer_sku: planned value cty.StringVal("standard") for a non-computed attribute
      - .network_profile[0].outbound_type: planned value cty.StringVal("loadBalancer") for a non-computed attribute
      - .network_profile[0].load_balancer_profile: attribute representing nested block must not be unknown itself; set nested attribute values to unknown instead
      - .network_profile[0].nat_gateway_profile: attribute representing nested block must not be unknown itself; set nested attribute values to unknown instead
2022-01-26T08:44:41.255+0200 [INFO]  ReferenceTransformer: reference not found: "var.use_existing_acr_name"
2022-01-26T08:44:41.255+0200 [INFO]  ReferenceTransformer: reference not found: "var.principal_id"
2022-01-26T08:44:41.255+0200 [INFO]  ReferenceTransformer: reference not found: "var.principal_id"
2022-01-26T08:44:41.255+0200 [INFO]  ReferenceTransformer: reference not found: "var.use_existing_acr_name"
2022-01-26T08:44:41.255+0200 [INFO]  ReferenceTransformer: reference not found: "var.storage_principal_id"
2022-01-26T08:44:41.428+0200 [WARN]  ValidateProviderConfig from "module.cluster.provider[\"registry.terraform.io/hashicorp/kubernetes\"]" changed the config value, but that value is unused
2022-01-26T08:44:41.435+0200 [WARN]  ValidateProviderConfig from "module.cluster.provider[\"registry.terraform.io/hashicorp/helm\"]" changed the config value, but that value is unused
2022-01-26T08:44:42.647+0200 [INFO]  ReferenceTransformer: reference not found: "var.enabled"
2022-01-26T08:44:42.647+0200 [INFO]  ReferenceTransformer: reference not found: "var.key_vault_sku"
2022-01-26T08:44:42.647+0200 [INFO]  ReferenceTransformer: reference not found: "local.tenant_id"
2022-01-26T08:44:42.647+0200 [INFO]  ReferenceTransformer: reference not found: "var.location"
2022-01-26T08:44:42.647+0200 [INFO]  ReferenceTransformer: reference not found: "local.key_vault_name"
2022-01-26T08:44:42.649+0200 [WARN]  Provider "registry.terraform.io/hashicorp/azurerm" produced an invalid plan for module.cluster.module.secrets.azurerm_key_vault.jx[0], but we are tolerating it because it is using the legacy plugin SDK.
    The following problems may be the cause of any confusing errors from downstream operations:
      - .soft_delete_retention_days: planned value cty.NumberIntVal(90) for a non-computed attribute
      - .network_acls: attribute representing nested block must not be unknown itself; set nested attribute values to unknown instead
2022-01-26T08:44:42.649+0200 [INFO]  ReferenceTransformer: reference not found: "var.enabled"
2022-01-26T08:44:42.649+0200 [INFO]  ReferenceTransformer: reference not found: "local.tenant_id"
2022-01-26T08:44:42.649+0200 [INFO]  ReferenceTransformer: reference not found: "var.enabled"
2022-01-26T08:44:42.649+0200 [INFO]  ReferenceTransformer: reference not found: "var.principal_id"
2022-01-26T08:44:42.649+0200 [INFO]  ReferenceTransformer: reference not found: "local.tenant_id"
2022-01-26T08:44:42.650+0200 [INFO]  ReferenceTransformer: reference not found: "var.job_secret_env_vars"
2022-01-26T08:44:42.650+0200 [INFO]  ReferenceTransformer: reference not found: "var.operator_annotations"
2022-01-26T08:44:42.650+0200 [INFO]  ReferenceTransformer: reference not found: "var.jx_git_url"
2022-01-26T08:44:42.650+0200 [INFO]  ReferenceTransformer: reference not found: "var.jx_bot_username"
2022-01-26T08:44:42.650+0200 [INFO]  ReferenceTransformer: reference not found: "var.jx_bot_token"
2022-01-26T08:44:42.650+0200 [INFO]  ReferenceTransformer: reference not found: "module.cluster"
2022-01-26T08:44:42.650+0200 [INFO]  ReferenceTransformer: reference not found: "local.jx_requirements_content"
2022-01-26T08:44:42.651+0200 [INFO]  ReferenceTransformer: reference not found: "azurerm_key_vault_access_policy.terraform"
2022-01-26T08:44:42.651+0200 [INFO]  ReferenceTransformer: reference not found: "local.secret_map"
2022-01-26T08:44:42.651+0200 [INFO]  ReferenceTransformer: reference not found: "each.key"
2022-01-26T08:44:42.651+0200 [INFO]  ReferenceTransformer: reference not found: "each.value"
2022-01-26T08:44:42.651+0200 [INFO]  ReferenceTransformer: reference not found: "azurerm_key_vault_access_policy.terraform"
2022-01-26T08:44:42.651+0200 [INFO]  ReferenceTransformer: reference not found: "local.secret_map"
2022-01-26T08:44:42.651+0200 [INFO]  ReferenceTransformer: reference not found: "each.value"
2022-01-26T08:44:42.651+0200 [INFO]  ReferenceTransformer: reference not found: "each.key"
2022-01-26T08:44:42.652+0200 [WARN]  Provider "registry.terraform.io/hashicorp/kubernetes" produced an invalid plan for module.cluster.kubernetes_config_map.jenkins_x_requirements, but we are tolerating it because it is using the legacy plugin SDK.
    The following problems may be the cause of any confusing errors from downstream operations:
      - .data: planned value cty.UnknownVal(cty.Map(cty.String)) does not match config value cty.MapVal(map[string]cty.Value{"jx-requirements.yml":cty.UnknownVal(cty.String)})

Jan 25 '22 14:01 royb-sw