document the CAPI cluster upgrades

[smira]

Creating new MachineTemplate, rolling out updates for the workers and control plane nodes.

[blackliner]

reading the template, can you confirm the following use-cases:

!!! NOT CONFIRMED YET !!!

I want to change the talos version that is used during PXE bootstrapping:

• Environments are mutable, so you can change spec.initrd.url and spec.kernel.url to the new talos version. Hint: Environments are referenced in a ServerClass' spec.environmentRef.name
  • Since CAPI does not support in place updates, no existing servers/nodes will be upgraded. Only if a new server is bootstrapped, it gets the new Talos Version. Alternative: use talosctl to manually upgrade Talos on each node.

I want to change the control plane talos version:

• TalosControlPlanes are mutable, so you can change spec.controlPlaneConfig.controlplane.talosVersion to the new talos version.
• observed issue: a node that was replaced will not automatically restart, is stuck and needs a manual reboot.
• conflicts with https://github.com/siderolabs/sidero/issues/1101#issuecomment-1503895801

I want to change the control plane kubernetes version:

• TalosControlPlanes are mutable, so you can change spec.version to the new k8s version.

I want to change the worker nodes talos version:

• TalosConfigTemplates are immutable, so a new one needs to be created an updated talos version at spec.template.spec.talosVersion, and linked in the existing MachineDeployment
• conflicts with https://github.com/siderolabs/sidero/issues/1101#issuecomment-1504986565
• conflicts with https://github.com/siderolabs/sidero/issues/1101#issuecomment-1503895801

I want to change the worker nodes kubernetes version:

• MachineDeployments are mutable, so you can change spec.template.spec.version to the new k8s version.