Terraform & Fortigate Reordering Error

[leonardoauribe]

Terraform and Fortigate are conflicting on the ordering of members in my service and address groups causing diffs and always appearing in my terraform plans.

TERRAFORM PLAN

  # module.policy_objects.fortios_firewallservice_group.dod_vip_svcgrp[0] will be updated in-place
  ~ resource "fortios_firewallservice_group" "dod_vip_svcgrp" {
        id                    = "dod-vip-ports"
        name                  = "dod-vip-ports"
        # (5 unchanged attributes hidden)

      ~ member {
          ~ name = "MTLS" -> "HTTPS_ALT"
        }
      ~ member {
          ~ name = "HTTPS_ALT" -> "MTLS"
        }
        # (2 unchanged blocks hidden)
    }

  # module.policy_objects.fortios_firewallservice_group.mail_protocols_svcgrp[0] will be updated in-place
  ~ resource "fortios_firewallservice_group" "mail_protocols_svcgrp" {
        id                    = "mail-protocols"
        name                  = "mail-protocols"
        # (5 unchanged attributes hidden)

      ~ member {
          ~ name = "SMTP_10025" -> "SMTP_587"
        }
      ~ member {
          ~ name = "SMTP_11025" -> "SMTP_10025"
        }
      ~ member {
          ~ name = "SMTP_587" -> "SMTP_11025"
        }
        # (1 unchanged block hidden)
    }

'''

The Fortigate takes the configuration and reorders it once applied. I keep getting the same output shown above over and over again. This is an issue because this code is applied to multiple Fortigates some of which have the ordering slightly different so If I fix it in code for one it will likely cause the issue in another. 

Is there a way for terraform to ignore the ordering or some other solution?

[lix-fortinet]

Hi @leonardoauribe,

Thank you for raising this issue. We will appreciate it if you could provide an example of your test case and the environment for your test case, like the FortiOS version, FortiOS Terraform provider version, Terraform version, etc. That would be very helpful for us to reproduce this issue.

Thanks, Xing

[leonardoauribe]

Version Details

Terraform v1.0.1
on darwin_amd64
+ provider registry.terraform.io/fortinetdev/fortios v1.14.0
+ provider registry.terraform.io/hashicorp/local v2.2.2

FortiOS version: v6.4.7

Let me know if more is required. thanks!

[lix-fortinet]

Hi @leonardoauribe,

Thank you for your update. Does the argument dynamic_sort_subtable set to true in your configuration? If so, the block arguments, like member, will be sorted on .tfstate file, which means the data get from FortiOS will be sorted. However, the sequence in Terraform configuration is not controlled by provider side. So, you may get this issue due to the sequence not matching. Team are working on this issue. But for now, you could try the following solutions:

1. Delete argument dynamic_sort_subtable or set it to false.
2. Make the sequence of member in nature order to match the sequence in .tfstate file.

Thanks, Xing

[leonardoauribe]

Argument 'dynamic_sort_subtable" is set to false on all resources. I went ahead and changed the member sequence on the Fortigates to match the sequence in the .tfstate file and that did the trick. I do still believe a solution should be put in place to where order doesn't matter. thanks for the help on this issue.

[lix-fortinet]

Hi @leonardoauribe,

Thank you for your comment. Team are working on the sorting issue. We will get back to you once we have any updates.

Thanks, Xing

[leonardoauribe]

It appears that this issue does still exist on some of our Fortigates. Has there been any progress made with fixing the sorting issue?

[kunstkomputer]

I can confirm, that we are seeing the same issue with another ressource. We are trying to manage fortios_firewall_addrgrp with roughly 50 members. We also observe, that objects are reordered by the provider or by API itself. Is there anything we can do in this case, we would be also happy to contribute.

[xav-git]

Hello, We also have the same problem. This issue generates a lot of confusion since it happens in many different FortiOS objects such as:

• fortios_firewal_ addrgrp
• fortios_firewallservice_custom
• fortios_user_group
• fortios_firewall_policy : srcaddr and dstaddr fields

So it seems any tables in FortiOS cli.

We believe the bug is in the terraform fortios provider because what happens is that upon a plan, a refresh is performed which correctly performs a GET via RESTAPI and updates the tfstate information as it is on the FortiGate. However although the order has changed in the tfstate during the refresh phase the provider ignores this change and does not PUT or POST the object with the new tfstate order definition.

Is seems therefore that a terraform refresh takes into account the order but not the terraform apply

This problem is very easy to reproduce, take any object list like groups or srcaddr in policy that has two or more items. apply this group to the FortiGate, then edit the tf file, change member orders and apply again: ==> It will never success and all following plan / apply will always report changes to be performed.

Note: this should be done with dynamic 'member' and dynamic_sort_subtable = false like in the below TF snapshot:

resource "fortios_firewall_addrgrp" "dummy_address_group" {
  vdomparam = "dcva"
  name   = "dummy_address_group"
  dynamic_sort_subtable = false

  dynamic "member" {
    for_each = [ fortios_firewall_address.host_a.name,
                 fortios_firewall_address.host_b.name, 
               fortios_firewall_address.host_c.name ]
    content { 
      name = member.value
    }
  }
}

Version details:

Terraform v1.1.7
on linux_amd64
+ provider registry.terraform.io/fortinetdev/fortios v1.17.0
+ provider registry.terraform.io/hashicorp/local v2.4.0

FortiOS version: v7.2.5

[lix-fortinet]

Hi,

Sorry that we forgot to update this issue. We made improvement when dynamic_sort_subtable set to true. If set dynamic_sort_subtable to true, user need to make sure the order on .tf file are sorted. For example, could use Terraform function sort sort the items:

resource "fortios_firewall_addrgrp" "dummy_address_group" {
  name   = "dummy_address_group"
  dynamic_sort_subtable = true

  dynamic "member" {
    for_each = sort([ fortios_firewall_address.trname[2].name,
                 fortios_firewall_address.trname[0].name, 
               fortios_firewall_address.trname[1].name ])
    content { 
      name = member.value
    }
  }
}

As for setting dynamic_sort_subtable to false, the provider will send the request with the order in .tf file. But the REST API did not reorder the list variable for PUT operation. We will talk with REST API team about this issue.

Thanks, Xing

[leonardoauribe]

Thanks for the update! Right now we have "dynamic_sort_subtable" set to false. I'll wait to see what you hear back from REST API team.

[vincent-at-forti]

Hello Xing,

Thanks for your insight. Just a question: Members in firewall addrgrp, firewall service custom group , user groups etc... should be unordered as we don't really care of orders for them. I believe the schema definition of these objects should be more : schema.TypeSet (https://developer.hashicorp.com/terraform/plugin/sdkv2/schemas/schema-types#typeset ) which are unordered instead of schema.TypeList ( https://developer.hashicorp.com/terraform/plugin/sdkv2/schemas/schema-types#typelist ) which are ordered.

And in that case , we could get rid of that reordering order as terraform would not care of order while planning the change.

Or am I misleading?

[lix-fortinet]

Hi Vincent,

Thank you for your suggestion! We will investigate it and improve it on next release.

Thanks, Xing

[leonardoauribe]

Any update on this?

[MaxxLiu22]

Hi @leonardoauribe ,

Sorry for the late update, this blocks reorder issue has been fixed on the latest terraform FOS provider 1.18.1, feel free to upgrade to it. Let me know if that doesn't solve your question.

Thanks, Maxx

[leonardoauribe]

thanks for the update! I will to get this provider upgraded.