canonical
Allow disabling "Joining server certificate SAN does not contain join token name" check
Summary
Allow disabling "Joining server certificate SAN does not contain join token name" check with config.
Description
Prior to microcluster v2, it was possible to do the equivalent of:
root@node1:~$ microctl bootstrap --name node1
root@node1:~$ microctl add-node something
Then have node2 join with (even though something != node2:
root@node2:~$ microctl join $token --name node2
This no longer works after bumping to microcluster v2, since it fails at https://github.com/canonical/microcluster/blob/335e20468c37cb2caccc683fad6d1ab85244be29/internal/rest/resources/cluster.go#L149-L152 (because something is not in the SANs that node2 presents when joining).
This breaks some deployment scenarios for the Canonical Kubernetes ClusterAPI provider, where we generate a join token that is then seeded into a cloud-init script for instances. Since the instance does not yet exist before the token is created, we cannot know the hostname, therefore the eventual join command fails. This set of requirements (we need to seed the token to the cloud-init, the hostname cannot be known at that time) are not something that can be challenged.
One way to work around this problem, would be to have some config option, e.g. InsecureSkipVerifyTokenName that would disable this check.
cc @canonical/kubernetes
In general, I'm not a fan of allowing a "skip verification" or "insecure" flag because this can lead to bad practices that we will be stuck working around. So before we consider that, I'd like to see if there is some other way to preserve the concept of a join token being restricted to exactly one system based on the initially provided data.
So I have a couple questions first:
-
Why is a join token being generated for a system that does not yet exist? Is it possible to change this flow so that the system is created first, then the join token is issued, and finally the token is supplied to the system?
-
Even if the system is created after issuing the join token, is there any other identifying information that can be supplied to the existing cluster when issuing the join token? On the Canonical K8s side, how does the cluster ensure that the join token is being used by the right system, and is protected against MITM attacks?
One way that we can ensure reasonable care has gone into verification of the join token is to extend the join token API to accept optional metadata in addition to the name field. Then we can expose a VerifyJoinToken hook that, if defined by Canonical K8s, will be used in lieu of the internal verification mechanism which checks the server.crt SAN against the token name. There is a chance that an extension like this may be a breaking change to the existing join token API, and thus may be limited to v3 rather than the LTS.
Why is a join token being generated for a system that does not yet exist? Is it possible to change this flow so that the system is created first, then the join token is issued, and finally the token is supplied to the system?
This is related to how ClusterAPI works. A node is created by calling the cloud/infrastructure API and supplying a cloud-init script. Since a created node needs to join the cluster, we need to generate the join token and seed/supply it into the cloud-init script beforehand. This is by design and the assumptions come from the upstream project. Our providers do not get access to the machines after they are created so the only time we can call commands and perform operations is in the cloud-init script generation phase. So it is not possible to change this flow.
Even if the system is created after issuing the join token, is there any other identifying information that can be supplied to the existing cluster when issuing the join token? On the Canonical K8s side, how does the cluster ensure that the join token is being used by the right system, and is protected against MITM attacks?
We could provide some sort of generated identifying information/string when generating a token and the same information can be seeded into the cloud-init script, however this does not feel any different than what is done today with just the token. One token per node is generated which is not posted/shared anywhere and it is directly supplied to the cloud-init script. I think the risks/vectors are the same for compromising this identifying string/information as well.
One token per node is generated which is not posted/shared anywhere and it is directly supplied to the cloud-init script.
When you create the token, you already provide a name. Later on when joining the node into the existing cluster you should use the same name to create the right certificate which will then also pass the check.
Can you not use those names internally (as expected by microcluster) and track the actual host name of the node separately?
