Clarification of errexit behaviour

[alza-bitz]

Hi there,

I noticed a difference in the behaviour of Bats recently, which I wanted to query to check I understand it correctly..

When I load external functions into a Bats test in order to test them, when I call them without run, the function has errexit on (i.e. set -e).

However, when I call the same functions with run, (e.g. to assert output or exit status) the function has errexit off.

Is this normal behaviour of Bats, and if so why the difference in the setting of errexit in these cases?

Thanks,

[ztombol]

TL;DR

Yes, you understood it correctly. run has errexit turned off. This is the intended behaviour and this is what allows testing commands that return non-zero status. Otherwise the non-zero status would automatically trigger the failure of a test.

Note: It doesn't matter if you load the tested command or it's in the test file. load is just a wrapper around source.

Details

By default, errexit is on, i.e. set -e, so that if a command returns non-zero status the test automatically fails. From the documentation (emphasis mine):

Test cases consist of standard shell commands. Bats makes use of Bash's errexit (set -e) option when running test cases. If every command in the test case exits with a 0 status code (success), the test passes. In this way, each line is an assertion of truth.

This, in my experience, is the most overlooked and brilliant feature of Bats. This enables the compact assertion syntax, i.e. just write an expression and the return value is automatically checked, and allows the use of the full power of Bash instead of restricting the user to a custom assertion language.

However, this behaviour, i.e. set -e, would make it diffcult to test commands that return non-zero status as it would trigger a test failure automatically. This is what the run command is for. From the documentation (emphasis mine):

Bats includes a run helper that invokes its arguments as a command, saves the exit status and output into special global variables, and then returns with a 0 status code so you can continue to make assertions in your test case.

run achives this by disabling errexit (i.e. set +e), running the command and saving the status while a non-zero status does not cause a test failure, and finally reenabling errexit (i.e. set -e).