Epic: Support ARM AWS instances

[stradig]

Motivation

ARM instances in AWS are in most cases cheaper than x86 instances. Thus to reduce COGS it makes sense to switch to ARM instances.

There are a couple of aspects which need to be taken into account when switching to ARM instances:

• A different method for adding and removing CPUs needs to be used because on ARM hotplugging CPUs is not supported
• Since at least at the moment our K8s clusters are used for running not only compute but also other kinds of services we must ensure that all those work on ARM. Or we need to use mixed clusters and make sure that only those services which are running fine on ARM are running there.
• Performance of ARM CPUs will most probably differ from x86 nodes. Since we use the notion of a CU we need to calibrate the actual amount of resources one CU is getting when running on ARM so that customers get a similar performance when switching to ARM.

DoD

• Computes can run on ARM instances without negative effects on the rest of the deployment, this includes autoscaling.
• Customers get similar performance when running on ARM than when running on x86