virt-test icon indicating copy to clipboard operation
virt-test copied to clipboard

./run: --list-tests: list disabled tests as well

Open ehabkost opened this issue 12 years ago • 3 comments

Today I got confused trying to get the "cpuflags" test case to run and not seeing it in the --list-tests output. After asking for help, I learned that the --list-tests output depend on the guest-os choice, and "cpuflags" has "no JeOS" in its config file.

What I request is that --list-tests lists the disabled test cases as well (clearly indicating that they are disabled), so we know the test case exists.

ehabkost avatar Jan 24 '13 19:01 ehabkost

I need to think about this one... some internal reorganization will be required for this.

lmr avatar Jan 24 '13 20:01 lmr

This is the only one that I could not implement in a straightforward way. The way we generate the list of tests is to parse the base config files without any test filtering, but with the guest OS filter already applied. Now, we could generate 2 sets of tests, one with the guest OS filter not applied, and other with the guest OS filter applied, and mark the difference as "Tests disabled on guest os type [guest-os-type]"... what do you say @ehabkost ?

lmr avatar Jan 25 '13 04:01 lmr

I guess it would work. I don't care too much about the actual implementation, I am just looking for a "user-visible model" that doesn't require understanding the cartesian config file. I like a lot the model exposed by ./run today, because it rarely requires understanding of the config file and cartesian products.

Maybe a solution is to make the config parsing code allow more "introspection". Instead of asking "give me the test dictionaries generated from subtests.cfg", we could ask it "give me the list of variants parsed on subtests.cfg (just the parser syntax tree, without any filtering or cartesian product calculation)".

ehabkost avatar Jan 25 '13 11:01 ehabkost