Some optional blocks in `auth0_connection` behave differently from each other

[ad8lmondy]

Checklist

• [X] I have looked into the README and have not found a suitable solution or answer.
• [X] I have looked into the documentation and have not found a suitable solution or answer.
• [X] I have searched the issues and have not found a suitable solution or answer.
• [X] I have upgraded to the latest version of this provider and the issue still persists.
• [X] I have searched the Auth0 Community forums and have not found a suitable solution or answer.
• [X] I agree to the terms within the Auth0 Code of Conduct.

Description

Hello, and thanks again for the provider!

I'm trying to pass config values into an auth0_connection, so a lot of the variables are optional. Hopefully this gets the idea across:

resource "auth0_connection" "auth0" {
  for_each = {
    for x in local.databases : x.name => x
  }

  name     = each.value.name
  strategy = "auth0"

  enabled_clients = []

  options {
    password_policy                = try(each.value.options.password_policy, null)
    brute_force_protection         = try(each.value.options.brute_force_protection, null)
    enabled_database_customization = try(each.value.options.enabled_database_customization, null)
    import_mode                    = try(each.value.options.import_mode, null)
    requires_username              = try(each.value.options.requires_username, null)
    disable_signup                 = try(each.value.options.disable_signup, null)
    configuration                  = try(each.value.options.configuration, null)
    set_user_root_attributes       = try(each.value.options.set_user_root_attributes, null)

    mfa {
      active                 = try(each.value.options.mfa.active, null)
      return_enroll_settings = try(each.value.options.mfa.return_enroll_settings, null)
    }

    password_history {
      enable = try(each.value.options.password_history.enable, null)
      size   = try(each.value.options.password_history.size, null)
    }
  }
}

You can see above that almost all variables are either taken from config, or filled with null, to indicate that we'll just use whatever the provider sets as the default.

The issue comes in with the blocks within the options block. Some of them, like the mfa block, work just fine with the above code; while the rest of them, including the password_history one, won't work with this setup. On terraform apply, it throws this error:

╷
│ Error: 400 Bad Request: Payload validation error: 'Expected type boolean but found type null' on property options.password_history.enable.
│ 
│   with auth0_connection.auth0["OEM-DB"],
│   on databases_auth0.tf line 1, in resource "auth0_connection" "auth0":
│    1: resource "auth0_connection" "auth0" {
│ 
╵

I go into more detail in the reproduction section, but this seems to be because the password_history block is being sent as an empty {} map, which fails the payload validation.

Expectation

My expectation is that I should be able to pass any of the objects below to my auth0_connection code, and it to behave "as expected" - that is: if I don't specify a block in the object, it's as if the block was not set in the auth0_connection: (I know these objects are YAML, but the structure is the same :sweat_smile: ):

  - name: no_password_history_no_mfa
    type: auth0
    options:
      brute_force_protection: true
      enabled_database_customization: false
      requires_username: true
      password_policy: "excellent"
      password_no_personal_info: true
  - name: no_mfa
    type: auth0
    options:
      brute_force_protection: true
      enabled_database_customization: false
      requires_username: true
      password_policy: "excellent"
      password_history:
        enable: true
        size: 3
      password_no_personal_info: true
  - name: no_password_history
    type: auth0
    options:
      brute_force_protection: true
      enabled_database_customization: false
      requires_username: true
      password_policy: "excellent"
      mfa:
        active: true
      password_no_personal_info: true
  - name: with_password_history_with_mfa
    type: auth0
    options:
      brute_force_protection: true
      enabled_database_customization: false
      requires_username: true
      password_policy: "excellent"
      password_history:
        enable: true
        size: 3
      mfa:
        active: true
      password_no_personal_info: true

At the moment, only no_mfa and with_password_history_with_mfa will work. The reproduction section (hopefully) explains why.

Reproduction

Commenting out both the mfa and password_history blocks in my auth0_connection, and creating a brand new DB with this object:

  - name: OEM-DB
    type: auth0
    options:
      brute_force_protection: true
      enabled_database_customization: false
      requires_username: true
      password_policy: "excellent"
      password_no_personal_info: true

gives

  # auth0_connection.auth0["OEM-DB"] will be created
  + resource "auth0_connection" "auth0" {
      + enabled_clients      = (known after apply)
      + id                   = (known after apply)
      + is_domain_connection = (known after apply)
      + name                 = "OEM-DB"
      + realms               = (known after apply)
      + strategy             = "auth0"
      + strategy_version     = (known after apply)

      + options {
          + allowed_audiences              = (known after apply)
          + brute_force_protection         = true
          + domain_aliases                 = (known after apply)
          + enabled_database_customization = false
          + ips                            = (known after apply)
          + non_persistent_attrs           = (known after apply)
          + password_policy                = "excellent"
          + requires_username              = true
          + scopes                         = (known after apply)
          + set_user_root_attributes       = (known after apply)
          + strategy_version               = (known after apply)

          + mfa {
              + active                 = (known after apply)
              + return_enroll_settings = (known after apply)
            }

          + password_complexity_options {
              + min_length = (known after apply)
            }

          + password_dictionary {
              + dictionary = (known after apply)
              + enable     = (known after apply)
            }

          + password_history {
              + enable = (known after apply)
              + size   = (known after apply)
            }

          + password_no_personal_info {
              + enable = (known after apply)
            }
        }
    }

and it is applied successfully

Making a single change: add the mfa block back into the auth0_connection:

    mfa {
      active                 = try(each.value.options.mfa.active, null)
      return_enroll_settings = try(each.value.options.mfa.return_enroll_settings, null)
    }

the plan looks like:

  # auth0_connection.auth0["OEM-DB"] will be updated in-place
  ~ resource "auth0_connection" "auth0" {
        id                   = "con_XYZ"
        name                 = "OEM-DB"
        # (5 unchanged attributes hidden)

      ~ options {
            # (27 unchanged attributes hidden)

          ~ mfa {
              - active                 = true -> null
              - return_enroll_settings = true -> null
            }
        }
    }

applying that also works fine.

Now, deleting the whole connection and this same process again, but instead of the mfa block, I'll add in the password_history block:

  # auth0_connection.auth0["OEM-DB"] will be updated in-place
  ~ resource "auth0_connection" "auth0" {
        id                   = "con_DEF"
        name                 = "OEM-DB"
        # (5 unchanged attributes hidden)

      ~ options {
            # (27 unchanged attributes hidden)

          + password_history {}

            # (1 unchanged block hidden)
        }
    }

but when I apply it, I get the error:

auth0_connection.auth0["OEM-DB"]: Modifying... [id=con_DEF]
╷
│ Error: 400 Bad Request: Payload validation error: 'Expected type boolean but found type null' on property options.password_history.enable.
│ 
│   with auth0_connection.auth0["OEM-DB"],
│   on databases_auth0.tf line 1, in resource "auth0_connection" "auth0":
│    1: resource "auth0_connection" "auth0" {
│ 
╵

I think the key is this difference:

          ~ mfa {
              - active                 = true -> null
              - return_enroll_settings = true -> null
            }

vs

          + password_history {}

the mfa block already exists on a brand new connection, whereas the password_history is a new thing - but since it contains 'nothing', the payload is wrong.

The work around I have at the moment is:

    dynamic "password_history" {
       for_each = contains(keys(each.value.options), "password_history") ? [null] : []      
       content {
        enable = each.value.options.password_history.enable
        size   = each.value.options.password_history.size
      }
    }

which is relatively compact, but quite ugly :sweat_smile:, and it seems that the mfa block is the only that doesn't need it.

Auth0 Terraform Provider version

v0.35.0

Terraform version

Terraform v1.2.7

[ad8lmondy]

Hmm, could be related to this issue: https://github.com/auth0/terraform-provider-auth0/issues/14

[ad8lmondy]

Kinda sorta related to https://github.com/auth0/terraform-provider-auth0/issues/314 too

[willvedd]

So just focusing just on the password_history block for now, the issue is that you're trying to pass the following payload:

password_history {
  enable = null
  size   =  null
}

This fails because the Management API does not allow the enable and size properties to be null. This is very clearly stated by the error message you received:

400 Bad Request: Payload validation error: 'Expected type boolean but found type null' on property options.password_history.enable.

Your expectation may be that the Terraform provider will magically omit the password_history (or any other) block when the values inside are completely or partially null, but that is not congruent with the way the provider operates. You are explicitly defining these values as null and there is no way to distinguish between intentionally null and intentionally omitted.

Admittedly, there are some minor DX enhancements that could be applied to validate and smooth-out what you're describing, but when it comes to sub-properties of blocks it becomes a bit more tricky. There are several different types of connection with their own bespoke options, some of which get minor changes and additions over time. We have intentionally erred on the side of maintaining flexibility and reducing false-negatives.

As for a solution, I recommend defining all possible values in your YAML and nix the try function, because the null defaults it is ultimately what is causing this. Otherwise, you would need to reformulate your connection resource definition to somehow not pass in null sub-values.

[ad8lmondy]

Hey, thanks for the reply!

Yes, I don't dispute your main point: "there is no way to distinguish between intentionally null and intentionally omitted." - this is what the dynamic block workaround... works around I guess :sweat_smile:

As you have laid it out, it does seem to be a relatively impossible request to fix (sorry about that ;)) - though my sticking point has been the mfa block, which seems happy enough to accept null even though it's not an appropriate boolean type.

In saying that, I do appreciate it's unclear how those nulls are being interpreted, in terms of the overall outcome, since there is no assurance they'd be interpreted the same as if the mfa block was omitted.

But it does make me curious: why is the mfa block the exception in how it behaves? Should it return the same error when passed nulls too?

[ad8lmondy]

I don't mean to re-open an older issue, but I came back to this just now, because the optional type stuff is now in Terraform.

It is relevant because you can specify optional fields to a variable, and I wondered how they handled the issue of addressing an optional entity within a variable when it was not provided - and it turns out it just sets the value to be null (you can see more here: https://www.terraform.io/language/expressions/type-constraints#example-nested-structures-with-optional-attributes-and-defaults )

From this discussion, I had thought: "how do you specify omitted vs intentionally null" - but looking here: https://www.terraform.io/language/expressions/types#null

null: a value that represents absence or omission. If you set an argument of a resource to null, Terraform behaves as though you had completely omitted it

Anyway - while that is interesting, this particular issue was about optionally specifying blocks - and it still appears that Terraform as a language does not support the same sort of idea as an optional block :(

Closing this off for now!