(as3)provider produced inconsistent final plan

[strantalis]

Environment

• TMOS/Bigip Version: v14.1.4
• Terraform Version: 0.13.6
• Terraform bigip provider Version: 1.7.0

Summary

When running an apply we are getting the below error and only see a small expected difference between what the as3 definition was and what it is now.

Error: Provider produced inconsistent final plan

When expanding the plan for module.loadbalancer.bigip_as3.loadbalancer to
include new values learned so far during apply, provider
"registry.terraform.io/f5networks/bigip" produced an invalid new value for

Change is removing a pool member and disabling the member.

 {
      "enable": false,
      "serverAddresses": [
          "X.X.X.X.X"
      ],
      "servicePort": 0,
      "shareNodes": true
  }

{
      "enable": false,
      "serverAddresses": [],
      "servicePort": 0,
      "shareNodes": true
  }

Steps To Reproduce

So at a high level we deployed a new as3 declaration which initially adds a pool member for us to bootstrap another product. Once that initial bootstrapping is complete we set a tfvar to remove that pool member and that is when the validation fails on the apply.

Expected Behavior

We expect the as3 definition to be posted to the f5 and not have failed on the validation stage.

Actual Behavior

Seems to fail comparing as3 definitions when there is a change.

[trinaths]

Can you provide some sample data to reproduce this issue? @strantalis

[strantalis]

@trinaths I opened a case so I could get you some data.

C3569531

[trinaths]

@strantalis that really helps. Thank you.

[trinaths]

@strantalis Can you try with simple AS3 changing the pool members ? we tried this, https://clouddocs.f5.com/products/extensions/f5-appsvcs-extension/latest/declarations/http-services.html#two-applications-sharing-a-pool and see no issue.

[strantalis]

I took the as3 we generate from the template function and ran it out side of our normal process and wasn't able to reproduce the issue either. Not sure how we hit this scenario or get ourselves out of it though.

[trinaths]

Ok. Can you reverify the same in your normal environment and share your findings. I think it must be your process issue.

[trinaths]

@strantalis - Is the AS3 declaration working well ?

[strantalis]

I need to spend sometime tomorrow looking at this. We are still hitting the issue in our pipeline but its difficult to debug if we can't replicate it outside of the normal process flow.

[trinaths]

@strantalis - any update on this issues?

[ajski1701]

We tested with provider versions 1.8.0 and 1.9.0 and got the same results. The provider seems to be only throwing the error when it is in the middle of comparing the AS3 the provider is going to apply versus the AS3 JSON the BIG-IP is currently configured with.

Most notably this seems to occur when the array serverAddresses in the pool members goes from a value of [ "w.x.y.z" ] to just an empty array of [].

[trinaths]

@ajski1701 @strantalis

1. use the below commands to delete the state of the of bigip_as3 module resource.

terraform state rm module.loadbalancer.bigip_as3.loadbalancer_1b
terraform state rm module.loadbalancer.bigip_as3.loadbalance

2. Delete any workarounds for this issue.

3. Change log level to debug.

4. share the logs from create and update of the resource.

[trinaths]

@strantalis - Is this a HA pair where TF module is used to update both Active and Standby BIGIP ?

[strantalis]

Yeah, our terraform applies the as3 configuration to both devices instead of them syncing with each other.

[papineni87]

@ajski1701 @strantalis

we are able to reproduce the issue when we manually change the expected json before apply stage i.e., plan shows server pool is empty but at the time of apply we will revert it back to original stage

steps to repro:

• Change the server pool to empty in as3 json
• Plan shows the expected diff ( some elements to null )
• Before terraform apply, revert back to old changes in as3 json( add some elements to server pool )
• Now terraform apply throws same error
• So it says " i am expecting server pool as empty , but i am seeing some elements "

So the issues looks more on rendering the as3 template which is not rendering as expected. This may not be issue with provider, as issue is more on rendering. Can we have static as3 template instead of rendering ?

[strantalis]

We build the servers dynamically within our terraform, so we really couldn't have a static as3 template.

[papineni87]

ok got it, but as i mentioned above we are able to reproduce the issue when the rendered as3 json differs in terraform Plan and Apply stages. So is there something else happening in the pipeline between the plan and apply stages ? Not sure why the as3 json is different from plan and apply stages.

[trinaths]

@strantalis - Please share logs generated using the debug build without any workaround.

[trinaths]

@strantalis - any update you can share ? logs with debug build without any workarounds.

[strantalis]

@trinaths I believe we uploaded a copy of the debug logs when originally opening the case.

[trinaths]

@strantalis - The new logs with debug build we shared from the SR.