Allow `-P` option to parameterise a fixture of a test

[vkarak]

Copying from the Slack discussion:

Indeed -P can only parameterize the leaf tests,  not the fixtures. Parameterizing the fixtures would require to redefine the whole test hierarchy from the fixture to the leaf nodes. I can’t think of a straightforward way of doing this atm. I would suggest opening a feature request for extending the behavior of the -P option to fixtures giving this example. The “how” needs a bit of thinking as ambiguities may need to be addressed, eg. a variable defined both in a fixture and a test higher up. (edited) 

[jack-morrison]

Copying the following here from Slack, in case that chat history ever goes away. The following message was my original inquiry related to this issue.

---

Is there a recommended way to parameterize over a variable used in both a test and a fixture of that test, keeping them consistent? We have an inheritance structure that looks somewhat like the following, taking an OSU microbenchmark as example:

              osu_alltoall(osu_run_base)
                    |
                    |
               osu_run_base(run_controls)
                    |
                    |- - - - - - - - - - - - - - - - - - - - - fixture: osu_build(spack_controls)
                    |                                                                              |
                    |                           spack_controls(rfm.CompileOnlyRegressionTest, multitest_controls)
                    |                                                                              |
    run_controls(rfm.RunOnlyRegressionTest, multitest_controls)
                    |                                                                              |
                    |                                                                              |
                    |                                                                              |
                    multitest_controls(rfm.RegressionMixin) ———
                     * variables used upwards through the build and run inheritance

Our aim is to create a few "lower level" layers to make authoring new tests simple. The root of our issue and question is when it comes to variables defined in multitest_controls.

With a variable VAR defined in multitest_controls, invoked with -P VAR=A,B the goal would be consistency between the run and build jobs. Something like:

osu_alltoall %osu_build.VAR=A
   ^ osu_build %VAR=A
osu_alltoall %osu_build.VAR=B
   ^ osu_build %VAR=B

However, when using -P, it appears we cannot parameterize in a fixture. Using -P, behavior looks sort of like this:

osu_alltoall %osu_build.VAR=A
  ^ osu_build %VAR=A
osu_alltoall %osu_build.VAR=B
  ^ osu_build %VAR=A

We've then tried changing VAR to a parameter type, but we're then seeing something like:

osu_alltoall %VAR=A
 ^ osu_build %VAR=A
 
osu_alltoall %VAR=A  --\
 ^ osu_build %VAR=B     | Undesirable
                        |
osu_alltoall %VAR=B     |
 ^ osu_build %VAR=A  --/
 
osu_alltoall %VAR=B
 ^ osu_build %VAR=B