Review CLI UX for `update`,`upgrade` and `utils`

[nikimanoledaki]

This ticket is to discuss, revisit and potentially improve the UX for update,upgrade and utils commands.

update vs upgrade

For context, we have been operating under the assumption that an update command should not/does not make any changes to resources. Rather, an update command will update configuration without necessarily making any changes. That would be the realm of upgrade.

There are a couple of inconsistencies about the meaning and difference between update and upgrade internally with some eksctl commands. For example, the following update commands actually do upgrades:

 eksctl update addon                           Upgrade an Addon

eksctl update iamserviceaccount --approve 

utils

A few utils commands actually do updates/upgrades:

  eksctl utils set-public-access-cidrs         Update public access CIDRs
  eksctl utils update-aws-node                 Update aws-node add-on to latest released version
  eksctl utils update-cluster-endpoints        Update Kubernetes API endpoint access configuration
  eksctl utils update-cluster-logging          Update cluster logging configuration
  eksctl utils update-coredns                  Update coredns add-on to ensure image matches the standard Amazon EKS version
  eksctl utils update-kube-proxy               Update kube-proxy add-on to ensure image matches Kubernetes control plane version

This would be a lot of work to refactor and change because they are likely used quite heavily, so we would need to deprecate them, make noise, etc.

The main issue with this, though, is that we are setting a precedent of adding and maintaining eksctl utils xyz, and then we risk going in the direction of adding more of these, for example, see: https://github.com/weaveworks/eksctl/issues/5301

We should revisit our update/upgrade UX logic so that we can be sure to add commands there instead of through utils.