kola: support test size requests

[cgwalters]

Followup to https://github.com/coreos/coreos-assembler/pull/1436

Most of our tests are happy with 512MB of RAM, but not all. I want to increase test speed by reducing RAM requests where we can by default. This will also make it potentially cheaper to do more testing in public cloud (e.g. on-demand t3.nano is 95% cheaper than our m4.medium default).

Relatedly we should probably scale processors with RAM size too, e.g. only provide 2 vCPUs for tests by default because over-provisioning vCPUs has a cost too.

Whenever we're working on our test framework we should also search for relevant prior art;

Debian autopkgtest-qemu requests 1GiB by default and as far as I can tell there's no standard mechanism for tests to request more (or less).

Fedora FMF has a YAML for this but it's specific to the provider.

[cgwalters]

Also related to this...we really need to be testing in the public clouds with a variety of instance types; see https://github.com/openshift/installer/issues/2919 - there are some notable differences between our default m4 type and more modern m5 types around network drivers, etc.

(And really the even bigger problem that we're hitting in OpenShift is not all regions have all instance types available so it's hard to have "a default")

[jlebon]

Related to this, we should make kola aware of its total CPU and RAM allowance when passed --parallel N on QEMU. E.g. some switch like --qemu-max-ram 16G should make kola schedule tests in a smart way so that it never goes over 16G.

[cgwalters]

This is now even more relevant since https://bugzilla.redhat.com/show_bug.cgi?id=1907030

xref https://github.com/coreos/fedora-coreos-config/commit/3dee27d

And to elaborate a specific problem is today we have 1024 hardcoded in both the pipelines and coreos-assembler, and any time we want to change it we need to change both, or we will subject ourselves to possible OOMs (or underutilization).

[jlebon]

So it looks like the primary RFE in this issue is for tests to request different RAM amounts. At least for QEMU, this is done. But we still need to wire minMemory for cloud tests.

Let's split out the bit in https://github.com/coreos/coreos-assembler/issues/1438#issuecomment-750364851 into a separate issue and see if anyone wants to pick that up (edit: https://github.com/coreos/coreos-assembler/issues/3118).