Terraform Plan Times Out

[spirrello]

When we run a Terraform plan against our LTM we're getting timeout errors. Is there a way to increase the timeout period? We're never able to establish a successful run of this script.

LTM 13.1.0.8 Build 0.0.3 Point Release 8 Provider version = "0.12.4" Terraform v0.12.6

5:57:08 + terraform plan -var-file=f5.secrets.tfvars -var-file=terraform.tfvars.json 15:57:09 [0m[1mRefreshing Terraform state in-memory prior to plan...[0m 15:57:09 The refreshed state will be used to calculate this plan, but will not be 15:57:09 persisted to local or remote state storage. 15:57:09 [0m 15:57:09
15:57:09 ------------------------------------------------------------------------ 15:58:17 [31m 15:58:17 [1m[31mError: [0m[0m[1mGet https://10.207.129.32/mgmt/tm/net/self: net/http: request canceled (Client.Timeout exceeded while awaiting headers)[0m 15:58:17
15:58:17 [0m on provider.tf line 5, in provider "bigip": 15:58:17 5: provider "bigip" [4m{[0m 15:58:17 [0m 15:58:17 [0m[0m

[spirrello]

By the way, I opened a case with F5 support and they asked the same question on if we can increase the timeout.

[dannyk81]

@spirrello really difficult to understand the output you've pasted, but this seems like you might have some connectivity issue between the machine running terraform and bigip.

are you able to run some manual curl commands against this bigip device? is port tcp/443 reachable on the bigip?

The above rest query just GETs the Self IPs, it shouldn't take more than a second, while in your case seems to timeout after ~1 minute (?)

This is the same rest query I did on a bigip device located >6000km from me:

$ curl -k -w "@curl-format.txt" -u admin:********* https://mad-bigip01.tuenti.mgm/mgmt/tm/net/self 
{"kind":"tm:net:self:selfcollectionstate","selfLink":"https://localhost/mgmt/tm/net/self?ver=12.1.1","items":....

    time_namelookup:  0.156028
       time_connect:  0.279073
    time_appconnect:  0.553400
   time_pretransfer:  0.553510
      time_redirect:  0.000000
 time_starttransfer:  0.754777
                    ----------
         time_total:  0.754919

:point_up: took less than 1 second overall

[spirrello]

We don't have any connectivity issues and I've verified this by running packet captures on the LTM's management interface. I've verified that the LTM is taking approximately 9 seconds to respond after the TLS handshake and the Terraform client resets the connection after finally getting a response from the LTM.

[dannyk81]

I've verified that the LTM is taking approximately 9 seconds to respond after the TLS handshake

9 seconds for a TLS handshake is an awful lot of time... this shouldn't be taking this long and I would try to understand the root cause of this.

In my example this takes 0.5 second (time_appconnect: 0.553400).

time_appconnect The time, in seconds, it took from the start until the SSL/SSH/etc connect/handshake to the remote host was completed.

---

In any case, the go-bigip library sets a default timeout of 60 seconds for all API calls https://github.com/f5devcentral/go-bigip/blob/cb399c531a76f0c01442fd50d78def40341fb8db/bigip.go#L17-L19

hard to read you capture screenshot, since there's no time reference (only deltas) and no direction (from/to) indications... but judging by the 59.8.. delta there in the middle, looks like the request is taking a lot longer than 9 seconds.

have you tried to run a curl for the same Rest query and measure how long it takes, a -w flag using a format file like this:

    time_namelookup:  %{time_namelookup}\n
       time_connect:  %{time_connect}\n
    time_appconnect:  %{time_appconnect}\n
   time_pretransfer:  %{time_pretransfer}\n
      time_redirect:  %{time_redirect}\n
 time_starttransfer:  %{time_starttransfer}\n
                    ----------\n
         time_total:  %{time_total}\n

can help diagnose where time is spent during the request.

[spirrello]

I have a case open with F5 and I'll share with what they come back with. Stay tuned.

[KrithikaChidambaram]

Hi, closing this request now. Please re-open if required or send an email to [email protected]. Thanks!