terraform-provider-bigip

terraform-provider-bigip copied to clipboard

Environment

• TMOS/Bigip Version: 15.1.5.1
• Terraform Version: 1.2.2
• Terraform bigip provider Version: 1.14.0

Summary

Destroying a bigiq license on the f5 in Azure times out. The bigip_common_license_manage_bigiq object waits until the F5 is responsive again, this does not happen when removing a license on Azure.

The same behavior is seen when removing the license on a local VE such as in Nuntanix, but the device starts responding again after a couple of minutes so this is not a big deal.

Steps To Reproduce

Using a bigiq unmanaged mondule such as:

resource "bigip_common_license_manage_bigiq" "bigip_license" {
  bigiq_address    = var.bigiq_ip
  bigiq_user       = var.bigiq_username
  bigiq_password   = var.bigiq_password
  license_poolname = local.license_name
  skukeyword1      = local.license_type
  assignment_type  = "UNMANAGED"
  unit_of_measure  = "yearly"
}

error after nearly 15 min (the timeout of terraform: Error: getting license revoking status from bigip failed with :Get "https://10.149.196.132/mgmt/tm/sys/license": dial tcp 10.149.196.132:443: i/o timeout (Client.Timeout exceeded while awaiting headers)

I have been waiting for the last 30 min and the device in Azure never comes back up after removing the license.

Expected Behavior

Terraform destroy should be able to continue with destroying the rest of the elements. Perhaps it would be nice to add a parameter to specify if you want to wait for confirmation, or a maximum time before accepting the license to be considered removed.

When using a simple script the license gets removed in about 8 seconds without waiting for the F5 to restart its services. The script I use is as followed:

BIGIQ_IP="$1"
BIGIQ_USER="$2"
BIGIQ_PASSWORD="$3"
LICENSE_POOL_NAME="$4"
BIGIP_IP="$5"
BIGIP_PASSWORD="$6"
TOKEN=$(curl -sk -H Content-Type: application/json -X POST https://$BIGIQ_IP/mgmt/shared/authn/login -d "{\"username\":\"$BIGIQ_USER\",\"password\":\"$BIGIQ_PASSWORD\",\"loginProviderName\":\"tmos\"}" | jq .token.token)
curl -sk -H "X-F5-Auth-Token: $TOKEN" -X POST https://$BIGIQ_IP/mgmt/cm/device/tasks/licensing/pool/member-management -d "{\"licensePoolName\":\"$LICENSE_POOL_NAME\",\"command\":\"revoke\",\"address\":\"$BIGIP_IP\",\"user\":\"admin\",\"password\":\"$BIGIP_PASSWORD\"}"

Actual Behavior

The current behavior waits until the F5 is responding again. This is a big issue since after deleting the license in Azure, the F5 is no longer reachablle.

Created [INFRAANO-812] for internal tracking

@Otimun is this issue reproducible, i tried in my local environment [its not azure environment]

Terraform will perform the following actions:

  # bigip_common_license_manage_bigiq.test_example will be destroyed
  - resource "bigip_common_license_manage_bigiq" "test_example" {
      - assignment_type       = "UNMANAGED" -> null
      - bigiq_address         = "xxxx.xxxx.xxx.xxx.xxx" -> null
      - bigiq_login_ref       = (sensitive value)
      - bigiq_password        = (sensitive value)
      - bigiq_token_auth      = (sensitive value)
      - bigiq_user            = (sensitive value)
      - device_license_status = "LICENSED" -> null
      - id                    = "<redacted>" -> null
      - license_poolname      = "myLicencePool" -> null
    }

Plan: 0 to add, 0 to change, 1 to destroy.
2022-09-01T14:40:29.768+0530 [DEBUG] command: asking for input: "\nDo you really want to destroy all resources?"

Do you really want to destroy all resources?
  Terraform will destroy all your managed infrastructure, as shown above.
  There is no undo. Only 'yes' will be accepted to confirm.

  Enter a value: yes

2022-09-01T14:40:32.624+0530 [INFO]  backend/local: apply calling Apply
2022-09-01T14:40:32.624+0530 [DEBUG] Building and walking apply graph for DestroyMode plan
2022-09-01T14:40:32.626+0530 [DEBUG] ProviderTransformer: "bigip_common_license_manage_bigiq.test_example (expand)" (*terraform.nodeExpandApplyableResource) needs provider["registry.terraform.io/f5networks/bigip"]
2022-09-01T14:40:32.626+0530 [DEBUG] ProviderTransformer: "bigip_common_license_manage_bigiq.test_example (destroy)" (*terraform.NodeDestroyResourceInstance) needs provider["registry.terraform.io/f5networks/bigip"]
2022-09-01T14:40:32.627+0530 [DEBUG] ReferenceTransformer: "bigip_common_license_manage_bigiq.test_example (expand)" references: []
2022-09-01T14:40:32.627+0530 [DEBUG] ReferenceTransformer: "var.partition_name" references: []
2022-09-01T14:40:32.627+0530 [DEBUG] ReferenceTransformer: "provider[\"registry.terraform.io/f5networks/bigip\"]" references: []
2022-09-01T14:40:32.627+0530 [DEBUG] pruneUnusedNodes: bigip_common_license_manage_bigiq.test_example (expand) is no longer needed, removing
2022-09-01T14:40:32.627+0530 [DEBUG] Starting graph walk: walkDestroy
2022-09-01T14:40:32.628+0530 [DEBUG] created provider logger: level=debug
2022-09-01T14:40:32.628+0530 [INFO]  provider: configuring client automatic mTLS
2022-09-01T14:40:32.639+0530 [DEBUG] provider: starting plugin: path=/Users/chinthalapalli/go/src/github.com/terraform-providers/terraform-provider-bigip/terraform-provider-bigip args=[/Users/chinthalapalli/go/src/github.com/terraform-providers/terraform-provider-bigip/terraform-provider-bigip]
2022-09-01T14:40:32.642+0530 [DEBUG] provider: plugin started: path=/Users/chinthalapalli/go/src/github.com/terraform-providers/terraform-provider-bigip/terraform-provider-bigip pid=66876
2022-09-01T14:40:32.642+0530 [DEBUG] provider: waiting for RPC address: path=/Users/chinthalapalli/go/src/github.com/terraform-providers/terraform-provider-bigip/terraform-provider-bigip
2022-09-01T14:40:32.660+0530 [INFO]  provider.terraform-provider-bigip: configuring server automatic mTLS: timestamp=2022-09-01T14:40:32.659+0530
2022-09-01T14:40:32.679+0530 [DEBUG] provider: using plugin: version=5
2022-09-01T14:40:32.679+0530 [DEBUG] provider.terraform-provider-bigip: plugin address: address=/tmp/plugin3660043796 network=unix timestamp=2022-09-01T14:40:32.679+0530
2022-09-01T14:40:32.692+0530 [DEBUG] provider.stdio: received EOF, stopping recv loop: err="rpc error: code = Unimplemented desc = unknown service plugin.GRPCStdio"
2022-09-01T14:40:32.695+0530 [DEBUG] No provider meta schema returned
2022-09-01T14:40:32.704+0530 [WARN]  ValidateProviderConfig from "provider[\"registry.terraform.io/f5networks/bigip\"]" changed the config value, but that value is unused
2022-09-01T14:40:32.704+0530 [DEBUG] provider.terraform-provider-bigip: 2022/09/01 14:40:32 [INFO] Initializing BigIP connection
bigip_common_license_manage_bigiq.test_example: Destroying... [id=test]
2022-09-01T14:40:33.828+0530 [INFO]  Starting apply for bigip_common_license_manage_bigiq.test_example
2022-09-01T14:40:33.828+0530 [DEBUG] bigip_common_license_manage_bigiq.test_example: applying the planned Delete change
2022-09-01T14:40:33.829+0530 [DEBUG] provider.terraform-provider-bigip: 2022/09/01 14:40:33 Revoke License assignment for :https://xxx.xxx.xxx.xxx
2022-09-01T14:40:33.829+0530 [DEBUG] provider.terraform-provider-bigip: 2022/09/01 14:40:33 [INFO] Initializing BigIP connection
2022-09-01T14:40:35.355+0530 [DEBUG] provider.terraform-provider-bigip: 2022/09/01 14:40:35 [INFO] revoke license to BIGIP device:10.145.71.31 from BIGIQ
2022-09-01T14:40:40.619+0530 [DEBUG] provider.terraform-provider-bigip: 2022/09/01 14:40:40 [DEBUG] wait for bigip status with license revoking
bigip_common_license_manage_bigiq.test_example: Still destroying... [id=8401b777-d812-44d2-b8e0-03c949b9ceb9, 10s elapsed]
bigip_common_license_manage_bigiq.test_example: Still destroying... [id=8401b777-d812-44d2-b8e0-03c949b9ceb9, 20s elapsed]
bigip_common_license_manage_bigiq.test_example: Still destroying... [id=8401b777-d812-44d2-b8e0-03c949b9ceb9, 30s elapsed]
bigip_common_license_manage_bigiq.test_example: Still destroying... [id=8401b777-d812-44d2-b8e0-03c949b9ceb9, 40s elapsed]
bigip_common_license_manage_bigiq.test_example: Still destroying... [id=8401b777-d812-44d2-b8e0-03c949b9ceb9, 50s elapsed]
bigip_common_license_manage_bigiq.test_example: Still destroying... [id=8401b777-d812-44d2-b8e0-03c949b9ceb9, 1m0s elapsed]
bigip_common_license_manage_bigiq.test_example: Still destroying... [id=8401b777-d812-44d2-b8e0-03c949b9ceb9, 1m10s elapsed]
bigip_common_license_manage_bigiq.test_example: Still destroying... [id=8401b777-d812-44d2-b8e0-03c949b9ceb9, 1m20s elapsed]
bigip_common_license_manage_bigiq.test_example: Still destroying... [id=8401b777-d812-44d2-b8e0-03c949b9ceb9, 1m30s elapsed]
bigip_common_license_manage_bigiq.test_example: Still destroying... [id=8401b777-d812-44d2-b8e0-03c949b9ceb9, 1m40s elapsed]
bigip_common_license_manage_bigiq.test_example: Still destroying... [id=8401b777-d812-44d2-b8e0-03c949b9ceb9, 1m50s elapsed]
bigip_common_license_manage_bigiq.test_example: Still destroying... [id=8401b777-d812-44d2-b8e0-03c949b9ceb9, 2m0s elapsed]
bigip_common_license_manage_bigiq.test_example: Still destroying... [id=8401b777-d812-44d2-b8e0-03c949b9ceb9, 2m10s elapsed]
bigip_common_license_manage_bigiq.test_example: Still destroying... [id=8401b777-d812-44d2-b8e0-03c949b9ceb9, 2m20s elapsed]
bigip_common_license_manage_bigiq.test_example: Still destroying... [id=8401b777-d812-44d2-b8e0-03c949b9ceb9, 2m30s elapsed]
bigip_common_license_manage_bigiq.test_example: Still destroying... [id=8401b777-d812-44d2-b8e0-03c949b9ceb9, 2m40s elapsed]
bigip_common_license_manage_bigiq.test_example: Still destroying... [id=8401b777-d812-44d2-b8e0-03c949b9ceb9, 2m50s elapsed]
2022-09-01T14:43:33.724+0530 [DEBUG] provider.terraform-provider-bigip: 2022/09/01 14:43:33 [INFO] License Revoking for Device https://10.145.71.31 Success
bigip_common_license_manage_bigiq.test_example: Destruction complete after 3m0s
2022-09-01T14:43:33.753+0530 [DEBUG] provider: plugin process exited: path=/Users/chinthalapalli/go/src/github.com/terraform-providers/terraform-provider-bigip/terraform-provider-bigip pid=66876
2022-09-01T14:43:33.753+0530 [DEBUG] provider: plugin exited

Destroy complete! Resources: 1 destroyed.

Hi @RavinderReddyF5,

Yes, so far I have only encountered this issue in Azure. We deploy similar F5s on Nutanix and they do not have this same issue. It seems that in Azure when you remove the license ythe VM is taken offline, because you also no longer are able to communicate to the device. (no ping, ssh or https)

As a workaround I use the script that is mentioned in my initial post. to be able to use it I have to remove the state: terraform state rm module.license.bigip_common_license_manage_bigiq.bigip_license and I call the script only during the destroy fase using the following terraform code:

resource "null_resource" "delete_license" {
  count = var.delete_script == true ? 1 : 0
  triggers = {
    bigiq_ip       = var.bigiq_ip
    bigiq_username = var.bigiq_username
    bigiq_password = var.bigiq_password
    license_pool   = local.license_name
    bigip_ip       = var.bigip_ip
    bigip_password = var.bigip_password
  }
  provisioner "local-exec" {
    when    = destroy
    command = "${path.module}/delete-license.sh ${self.triggers.bigiq_ip} ${self.triggers.bigiq_username} ${self.triggers.bigiq_password} ${self.triggers.license_pool} ${self.triggers.bigip_ip} ${self.triggers.bigip_password}"
  }
  depends_on = [bigip_common_license_manage_bigiq.bigip_license]
}

But to answer your question, yes it is reproducable. My automation tests keep on having the same issue if I do not use this workaround.

@Otimun did u try using this config, as device is deployed in azure, it might be unreachable device

 # UNREACHABLE Regkey Pool
resource "bigip_common_license_manage_bigiq" "test_example" {
  bigiq_address    = "xxx.xxx.xxx.xxx"
  bigiq_user       = "xxxx"
  bigiq_password   = "xxxxx"
  license_poolname = "regkey_pool_name"
  assignment_type  = "UNREACHABLE"
  mac_address      = "FA:16:3E:1B:6D:32"
  hypervisor       = "azure"
}

Hi, closing this request now. Please re-open if required or send an email to [email protected]. Thanks!