support Distros key in JSON headers of external tests

[miabbott]

In openshift/os#661 it was discussed that some of the tests that are present in FCOS are not being shared/run by RHCOS, since the symlinks from the fedora-coreos-config submodule to the tests directory are missing.

@jlebon proposed extending the JSON headers of the external tests to include support for the same Distros key that is used in the native kola tests.

This way we could symlink the entire fedora-coreos-config/tests directory in openshift/os and be able to sanely share the tests between the two projects.

[HuijingHei]

Thanks @miabbott, this sounds reasonable

[miabbott]

The support for this looks like it is already in place:

https://github.com/coreos/coreos-assembler/blob/main/mantle/kola/harness.go#L668-L677

https://github.com/coreos/coreos-assembler/blob/main/mantle/kola/harness.go#L879-L883

As a quick test, I added a kola.json header to the misc-ro test in openshift/os:

$ git diff tests/
diff --git a/tests/kola/misc-ro/misc-ro.sh b/tests/kola/misc-ro/misc-ro.sh
index 1d72017..77277b9 100755
--- a/tests/kola/misc-ro/misc-ro.sh
+++ b/tests/kola/misc-ro/misc-ro.sh
@@ -1,5 +1,6 @@
 #!/bin/bash
 # This test is a dumping ground for quick read-only tests.
+# kola: {"distros": "rhcos"}
 set -xeuo pipefail
 
 cd $(mktemp -d)

Then kola list showed the new distro value:

$ kola list | grep misc-ro
ext.config.misc-ro.misc-ro.sh                                   [all]                                                           [all]   [rhcos]

Without the header:

$ kola list | grep misc-ro
ext.config.misc-ro.misc-ro.sh                                   [all]                                                           [all]   [all]

[saqibali-2k]

The support for this looks like it is already in place

Looks like only the docs need to be updated here: https://github.com/coreos/coreos-assembler/blob/main/docs/kola/external-tests.md#kolajson

[miabbott]

I'm currently investigating the existing FCOS tests and the feasibility of being able to run them on RHCOS. I'll make a PR to fedora-coreos-config with the necessary distro keys in the near future. Then change how openshift/os is consuming/using the tests from FCOS.