Redesign connection-client association to be controlled via its own resource (instead of only by the connection)

[dputnamfr]

Checklist

• [X] I agree to the terms within the Auth0 Code of Conduct.

Describe the problem you'd like to have solved

We have a fairly static set of Auth0 connections from which we draw bodies of users. Adding new bodies of users is fairly rare. Ergo, we have a single Terraform root module that controls each Auth0 connection.

We dynamically generate Auth0 applications/clients. Adding new templated Auth0 application/clients is fairly regular. Ergo, we have a single Terraform root module that can be run in multiple tf workspaces with different variables to generate a (theoretically) infinitely scalable number of application/clients.

Each application/client should be associated with a dynamic combination of connections, dependent on the variables passed to the application/client. For example, I might spin up an application/client that should only be reachable by two of my customers, and I might spin up a near-identical application/client that should only be reachable by one of my customers and our internal team.

Currently, there is no way for our lovely application/client terraform module to automatically associate the brand spanking new application with an existing connection. One has to spin up the application and either manually set the 'toggle' slider from the application/client's 'connections' tab in the dashboard, OR one has to edit the connections terraform root module and add the client_id to the enabled_clients argument for the connection.

Why is this bad?

1. Manual action sucks, so that rules out the dashboard like right away.
2. Having to tweak two terraform modules in a specific order is a pretty irritating barrier to automation. It means two git pushes.
3. Deleting an application/client before it was removed from the 'connections' terraform module means the 'connections' terraform module would immediately become invalid (because you better believe I'm not hardcoding client_ids, I'm using an auth0_client datasource) and would need edited and the offending client removed (which is also hard because THE APPLICATION IS ALREADY GONE and the data source that refers to it is now invalid. We're talking some direct terraform state work if the application destruction wasn't done in a perfectly precise order).
4. Codified methods of determining what application/clients are reachable by what users is particularly difficult. I have to go through each connection terraform remote module and figure out what each set of users has access to, then filter that by the application I care about... instead of simply asking the application.
5. All companies on the planet have 'legacy' junk that was set up manually by some yahoo, right? We're no different. We have some 'connections' which are not terraformed. I can't edit these connections via terraform for bureaucracy reasons, which means I have no codifiable recourse to attaching new application/clients to them!

In short, managing the association between application/clients and connections must be done in multiple places in a specific order... and it shouldn't be that way! This really hamstrings the usefulness of Terraform in general, where a single terraform apply can bring up all the resources necessary for a specific modular subset of your overall infrastructure.

Describe the ideal solution

Taking a page from the AWS provider's book, let's look at how they handle aws_security_groups, which are a resource that be thought of akin to a grouping of firewall rules. "If the connection comes from the 10.0.0.0/8 CIDR, allow it, but if it comes from 10.1.0.0/16, deny it." This is a resource - like your auth0_connection - that is often associated with several other resources (rules) - like your auth0_clients.

https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group

At first glance, the security group is set very similar to allowed clients in your connections resource - a list of 'ingress' and 'egress' rules. However, the AWS provider also provides the following resource:

https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule

This resource allows you to specify an arbitrary rule and assign it to an existing security group. If you had a central security group (that controlled access to, say, a virtual machine) and created a fresh resource that should have access to the security group, you could simply create an aws_security_group_rule resource and associate it to the security group INSTEAD of having to hunt down the existing aws_security_group's terraform module and plug in a data source (that can't be guaranteed to continually exist).

Using this method, one generates an empty security group in a single terraform module (call it module A) (much like generating an unassociated auth0_connection) that protects an asset with a 'deny' flag, and then any time someone creates a terraform module (call it module B) with resources that should be allowed to pass the security group, module B simply adds the aws_security_group_rule resource to the security group built in module A. Module B can be templated and run in many different terraform workflows to always create a rule and always plug it into the security group from A, without ever needing to edit and re-run A... and removing a rule is as simple as destroying module B.

I would love to see an auth0_connection_client_association resource that follows this same strategy.

Alternatives and current workarounds

Manually doing things in the dashboard, in the cases where I cannot edit a connection via terraform (ie, data source only)

Additional context

Kudos to you for working on the provider in general!

[ad8lmondy]

I agree that having the association on the client side would make things a bit easier, so I'm all for that.

I've managed to implement that myself, which might be helpful to you?

I have the config loaded in from a yaml file, like this:

applications:
  - name: Auth0 Deploy
    app_type: non_interactive

  - name: ABC
    app_type: native

    database_connections:
      - user-pool-A

  - name: DEF
    app_type: native
    
    database_connections:
      - user-pool-A
      - user-pool-B

databases:
  - name: user-pool-A
    type: auth0

  - name: user-pool-B
    type: auth0

(with a lot of details omitted, of course :smile: ) Notably, the database connections are specified per client.

Once it gets loaded into a local variable (or you could skip the yaml, and just start with it all in TF), I do this:

locals {
  enabled_db_connections = transpose({  # flip the association, so that now the map is { db-name : [ client-name, client-name ] }
    for app in local.applications :
    app.name =>
    app.database_connections
  })

  # Because of yaml, we need to look up the actual ID of the client, based on their name. This block does this for us
  enabled_db_connections_ids = {
    for db, app_names in local.enabled_db_connections :
    db => [
      for app_name in app_names :
      module.application[app_name].application.client_id
    ]
  }
}

So then local.enabled_db_connections_ids looks like:

{ "user-pool-A": [ "abcdef", "hjiklm"],  "user-pool-B": [ "hjiklm"] }  # I've just made up the client IDs here

and you can pass this to a for_each or similar to construct the databases.

You can also detect if someone has asked for a DB which does not exist by adding this block to the locals:

  invalid_db_connections = setsubtract(
    keys(local.enabled_db_connections),
    local.databases[*].name
  )

and then using a pre-condition on an output:

output "ensure_no_invalid_db_connections_in_config" {
  value = local.invalid_db_connections

  precondition {
    condition     = length(local.invalid_db_connections) == 0
    error_message = "Application trying to add a db_connection that doesn't exist:\n${join(",\n", local.invalid_db_connections)}"
  }
}

Hope that helps! But also: it would be nice not to need to do this too :wink:

[biomath-vlad]

@ad8lmondy This is a neat trick, unfortunately I can't see how it is going to work when there are multiple application each one of them controlled by it's own terraform stack.

[drhagen]

This feature would be super useful for us. Right now, we have a script that runs after terraform that hits the API directly to add the newly created clients to the existing connection. It is...not our best piece of infrastructure.

If it would help the implementation, can I propose a design:

resource "auth0_client_connection_association" "my_client_connection" {
  client_id = auth0_client.my_client.id
  connection_id = auth0_connection.my_connection.id  # real world would get this from `data` or something
}

This is quite similar to auth0_client_grant that already exists and the various "associations" in the AWS terraform library, like aws_route_table_association.

[sergiught]

Hey folks, really appreciate the great explanation on this issue and the healthy discussion that followed.

I completely agree with you that having a specific resource that would take care of the associations between client and connection would greatly improve the developer experience and simplify managing complex configurations.

We'll discuss internally how to prioritize this work as soon as possible and circle back here to give you an update.

[sergiught]

Hey folks, I'm gonna start working on this next week and hopefully have it ready within the next release. I'll give you an update if there are any issues encountered, so we can discuss approaches that would suit you. Stay tuned!

[sergiught]

👋🏻 Hello all, I have this ready in https://github.com/auth0/terraform-provider-auth0/pull/379. Feedback is much appreciated!

[sergiught]

Closing this down as this is now available in the latest release: https://registry.terraform.io/providers/auth0/auth0/latest/docs/resources/connection_client.

[jwhitaker-swiftnav]

Does this change mean that to control the default DB and Google OAuth client, I need to go create resources for and manually import every combination of client+application in my tenant?

[sergiught]

Hey @jwhitaker-swiftnav 👋🏻

That's correct, you'll need to have a separate connection client resource association for every enabled client on your connection.

Example:

resource "auth0_connection" "my_conn" {
  name     = "My-Auth0-Connection"
  strategy = "auth0"
}

resource "auth0_client" "my_client" {
  name = "My-Auth0-Client"
}

resource "auth0_connection_client" "my_conn_client_assoc" {
  connection_id = auth0_connection.my_conn.id
  client_id     = auth0_client.my_client.id
}

For already existing associations you can import them by specifying the connection ID and client ID separated by :.

Example:

terraform import auth0_connection_client.my_conn_client_assoc con_XXXXX:XXXXXXXX