virt-test
virt-test copied to clipboard
./run: --list-tests: list disabled tests as well
Today I got confused trying to get the "cpuflags" test case to run and not seeing it in the --list-tests output. After asking for help, I learned that the --list-tests output depend on the guest-os choice, and "cpuflags" has "no JeOS" in its config file.
What I request is that --list-tests lists the disabled test cases as well (clearly indicating that they are disabled), so we know the test case exists.
I need to think about this one... some internal reorganization will be required for this.
This is the only one that I could not implement in a straightforward way. The way we generate the list of tests is to parse the base config files without any test filtering, but with the guest OS filter already applied. Now, we could generate 2 sets of tests, one with the guest OS filter not applied, and other with the guest OS filter applied, and mark the difference as "Tests disabled on guest os type [guest-os-type]"... what do you say @ehabkost ?
I guess it would work. I don't care too much about the actual implementation, I am just looking for a "user-visible model" that doesn't require understanding the cartesian config file. I like a lot the model exposed by ./run today, because it rarely requires understanding of the config file and cartesian products.
Maybe a solution is to make the config parsing code allow more "introspection". Instead of asking "give me the test dictionaries generated from subtests.cfg", we could ask it "give me the list of variants parsed on subtests.cfg (just the parser syntax tree, without any filtering or cartesian product calculation)".