unison icon indicating copy to clipboard operation
unison copied to clipboard

io.test does not fail if term doesn't exist

Open ceedubs opened this issue 1 year ago • 0 comments

Describe and demonstrate the bug

In a transcript if you call io.test and a test fails, then it errors with a non-zero exit code. However, if you call io.test with a term that doesn't exist, it returns a success (0) exit code. This can mask errors (such as in CI) where you think that tests are passing but in fact they aren't running.

Input:

```ucm
fresh/main> io.test doesNotExist
```

Output:

``` ucm
fresh/main> io.test doesNotExist

    New test results:
  
  😶 No tests available.

```

You can see in the output that it detects that there are no tests. What you can't see from the transcript output is that the transcript command exits with 0.

I think that the following should all probably be considered errors:

  • Passing in a nonexistent term
  • Passing in a term with the wrong type
  • Zero tests running (even if the term exists and has the right type, if it returns an empty test result list then something is probably off). But I'm less sure about this one.

Environment (please complete the following information):

  • ucm --version 572943993
  • OS/Architecture: x86 Linux

Additional context Add any other context about the problem here.

ceedubs avatar Oct 08 '24 14:10 ceedubs