quint
quint copied to clipboard
CLI should produce different error codes
Currently, if a test in cli-tests.md returns a non-zero exit code, we do not know, whether it was the expected behavior or not. For instance, if we expect the simulator to find a violation, and the tool is passed incorrect CLI arguments, the test is considered as passed, though it should not.
We should differentiate exit codes that are caused by different exit paths. For instance, all CLI parsing should return one exit code and quint run
and quint test
should return another error code.
Just wanted to express interest in this, it would be helpful to be able to tell whether an invariant violation was found or not just from the exit code not only for testing, but also for user-side tooling :)
I've added it on my stack