pyutilib icon indicating copy to clipboard operation
pyutilib copied to clipboard

Allow factorial specification of tests based on sets of options

Open whart222 opened this issue 8 years ago • 1 comments

The pyutilib.testdriver should allow specifying multiple sets of options for a solver. It would then run a "full-factorial" suite of tests. For example:

  solvers:
    solver_a:
      np:
        - 1
        - 2
        - 4
      strategy:
        - dive
        - pseudocost
    solver_b:
      np: 1
  problems:
    problem_1:
    problem_2:

would specify 14 tests.

This is on the wishlist because this capability has lots of difficult implications, including how it gets implemented, how fractional specifications are managed (in the 'tests' block), and how tests get named. If this was spelled out in more detail, I'd be willing to look into this. However, this might be a nice extension for John or Dave to dig into to help familiarize themselves with this code.

It is an open question how to consistently name the individual tests.

whart222 avatar Nov 15 '15 02:11 whart222