Terraform Plan Failing with "connection is shut down" Errors

[paprockiw]

Hey there, I've been working on using the big_ip provider to apply changes to an F5 system, and I've been running into issues on F5 versions 11.6 and 12.1. I'm using Terraform 0.11.3, and I've tried making the changes the big_ip provider, starting with 0.12.4 and even tried the latest version, 1.2.1 to see if I got different results, and I did not.

The problem is that when I run terraform plan, it processes a bunch of the resources, and then stops being able to connect and crashes at different points in the process. I'm using this to manage pools, nodes, and vips, but somewhere in there, it always breaks part way through, and I get an error message of "Errors: [connection is shut down]", which applies to the rest of the resources in the configuration after processing a bunch of them (seemingly) successfully. This is typically accompanied by a "unexpected EOF" error as well.

The logs look something like this with the names obscured for privacy purposes :

• module.f5.module.nodes.bigip_ltm_node.NODE_NAME: 1 error(s) occurred:
• module.f5.module.nodes.bigip_ltm_node.NODE_NAME: bigip_ltm_node.NODE_NAME: connection is shut down

2020/07/10 21:15:22 [ERROR] root.f5.vips: eval: *terraform.EvalRefresh, err: bigip_ltm_virtual_server.SERVER_NAME: unexpected EOF 2020/07/10 21:15:22 [ERROR] root.f5.vips: eval: *terraform.EvalSequence, err: bigip_ltm_virtual_server.SERVER_NAME: unexpected EOF 2020/07/10 21:15:22 [TRACE] [walkRefresh] Exiting eval tree: module.f5.module.vips.bigip_ltm_virtual_server.SERVER_NAME 2020/07/10 21:15:22 [TRACE] root.f5.nodes: eval: *terraform.EvalSequence 2020/07/10 21:15:22 [TRACE] root.f5.nodes: eval: *terraform.EvalGetProvider 2020/07/10 21:15:22 [TRACE] root.f5.nodes: eval: *terraform.EvalReadState 2020/07/10 21:15:22 [TRACE] root.f5.nodes: eval: *terraform.EvalSequence 2020/07/10 21:15:22 [TRACE] root.f5.nodes: eval: *terraform.EvalGetProvider 2020/07/10 21:15:22 [TRACE] root.f5.nodes: eval: *terraform.EvalReadState 2020/07/10 21:15:22 [TRACE] root.f5.nodes: eval: *terraform.EvalRefresh 2020-07-10T21:15:22.287Z [DEBUG] plugin: plugin process exited: path=/home/terraform-provisioning/.terraform/plugins/linux_amd64/terraform-provider-bigip_v0.12.4_x4

I've read about these two related error messages, which occur in other plugins as well. It seems to happen often in Terraform 0.11.3. It looks to be related to how Terraform processes values read from F5, such as the profiles or descriptions attached to various objects, but this is very hard to track down since the plan is failing at random places each time.

Is there any known fix for this that I'm missing? Perhaps upgrading to a newer version of Terraform or or the big_ip provider, or are we looking at a bug?

Thanks.

[papineni87]

I didn't get much info from the logs provided, but to single out the issue can you try below things

1. Update terraform-bigip-provider to 1.2.1 and terraform version to 0.12
2. Create just single node ( bigip_ltm_node )and check whether any errors showing up or not. We can then increment resources and see where the issues comes out.

We didn't see any basic issues with latest release (1.2.1 ) against bigip versions 13,14,15

[paprockiw]

So, I was able to tease out more information from the logs. There's a virtual server configuration that's throwing an error when it attempts to read from the iRESTController API. I don't have a copy of the logs anymore, unfortunately, but it was throwing an error on this method: bigip.resourceBigipLtmVirtualServerRead

The issue was similar to the one described here: https://github.com/DealerDotCom/terraform-provider-bigip/issues/28

In my case, we just took that virtual server configuration out of the terraform state, so that we aren't managing it through Terraform anymore, and everything worked again. We'll try to reintroduce the config again later and see if we can get it working. If we find any further information, I'll let you know.

[paprockiw]

I have some further information. We took the offending virtual server object out of the TF state to get things working again. We tried to re-import it and got this (I replaced the virtual server name with VIP_NAME here):

module.f5.module.vips.bigip_ltm_virtual_server.VIP_NAME: Importing from ID "/Common/VIP_NAME"... module.f5.module.vips.bigip_ltm_virtual_server.VIP_NAME: Import complete! Imported bigip_ltm_virtual_server (ID: /Common/VIP_NAME) module.f5.module.vips.bigip_ltm_virtual_server.VIP_NAME: Refreshing state... (ID: /Common/VIP_NAME) Error: module.f5.module.vips.bigip_ltm_virtual_server.VIP_NAME (import id: /Common/VIP_NAME): 1 error(s) occurred:

• import module.f5.module.vips.bigip_ltm_virtual_server.VIP_NAMEresult: /Common/VIP_NAME: bigip_ltm_virtual_server.VIP_NAME: unexpected EOF

[KrithikaChidambaram]

Hi, closing this request now. Please re-open if required or send an email to [email protected]. Thanks!