smoke icon indicating copy to clipboard operation
smoke copied to clipboard

Support for a way to run the same kind of test on many files

Open SamirTalwar opened this issue 5 years ago • 3 comments

One use case for Smoke is to simply provide many different potential input and output files, and compare them. However, it gets old doing the following:

tests:
  - name: thing 1
    stdin:
      file: thing1.in
    stdout:
      file: thing1.out

  - name: thing 2
    stdin:
      file: thing2.in
    stdout:
      file: thing2.out

  - name: thing 3
    stdin:
      file: thing3.in
    stdout:
      file: thing3.out

# ...

Not only is it monotonous, but error-prone.

@jaspervdj suggested finding a better way to avoid this repetition, just like goldplate does.

SamirTalwar avatar Jul 26 '20 16:07 SamirTalwar

Perhaps we could do something like this:

tests:
  - name: %
    stdin:
      file: *.in
    stdout:
      file: %.out

This would detect the glob expression in stdin.file, and expand it to the long form above, using % as the placeholder for the pattern matched by *.

SamirTalwar avatar Jul 26 '20 16:07 SamirTalwar

Alternatively, we could look at Dhall as a way of programmatically generating configuration files safely.

SamirTalwar avatar Jul 26 '20 16:07 SamirTalwar

Using Dhall would be quite nice, indeed.

jonaprieto avatar Feb 23 '23 13:02 jonaprieto