new-testing-script feature requests

[zilinc]

1

Run several tests at once. It requires extending the --only flag.

2

When the expected result is wip, the log doesn't tell me if the test has passed or failed. E.g. Now it shows:

tests/func/wip_0-ary-function.cogent: WIP (Pass by defualt)

What I want is something like:

tests/func/wip_0-ary-function.cogent: passed (but ignored as it's WIP)

or

tests/func/wip_0-ary-function.cogent: failed (but ignored as it's WIP)
.......
.... ERROR LOG GOES HERE ...
.......

3

Tests expected to fail should fail in the stage the config file specifies. E.g., when I give expected_result: fail and -flags: -g, if a test fails at a different stage other than in code-generation, then it should signal it as a FAIL instead of a PASS.

4

Something which can tell me which files in the tests directory are not yet covered in any of the test scripts.

[emmet-m]

1. and 2. should be straightforward. 3. However will require the compiler to return a different error code per phase (or parse the stdout), which is more involved

[zilinc]

1. and 2. should be straightforward. 3. However will require the compiler to return a different error code per phase (or parse the stdout), which is more involved

I'm happy with different error codes from the compiler.