Is there an easier way to stub a specific command or script with this `bats-mock`?

[dragon788]

trafficstars

I love the syntax of your implementation and it makes a lot of sense to me that the mock is a test double/spy that watches all the invocations and allows you to assert that it was called in specific ways, without necessarily having to write out an assertion for every single call.

While trolling StackOverflow and related sites I came across a question about mocking a script and checking the calls using bats-mock, but they were using the other version and it doesn't make it quite as easy to assert mock_called_with_args or similar, so I wrote up an answer with an example using your bats-mock, but I was having trouble emulating the stub behavior of actually shadowing/preempting a binary in the PATH so that the ${mock} definition was called instead. https://stackoverflow.com/q/38315185/3794873

I eventually came up with a hacky method by doing the mock creation in the setup() method or in each test and then by making a symlink to the mock's file path in the same directory and prepending that directory to the PATH variable and then unlinking and stripping the location back out of the PATH in the teardown or the end of the test. Is there a cleaner way to do this without repeating the code across all of the tests and in such a way that it allows for one or more binaries to be stubbed, in the case you want to avoid any external resources being accessed and you could stub success for the list?

Perhaps this is something that could be added to this better maintained version of bats-mock? Maybe something like mock_set_command_stub_name.

[dragon788]

Upon reflecting on this I also realized that you need to use an additional $(create_mock) call for each binary you want to stub, otherwise you won't be tracking the right number of calls and args, so making a helper function like what jasonkarns/bats-mock uses for stub is probably the way to go.

[grayhemp]

@dragon788 thanks very much for your feedback. I was thinking about possibilities of mocking executables by their real names myself. And yes, symlink-ing + PATH manipulations occurred to me as well. However, the issue with this approach is that the executable can be specified with a path component (like /usr/local/bin/psql) in the script and so the idea breaks here. Not speaking about possible PATH manipulations by the script itself.

I have a hint of idea of solving that by chroot-ing into a disposable tree and doing unsafe things there but nothing is clear here yet, so I'm not sure if it's going to be the solution or there will be something else. I was a little bit hesitant about this enhancement, but after your message I started seeing a clear demand in it, so I scheduled work on it in terms of v2.0.

[dragon788]

Was interesting checking the implementation of stub.sh after discovering it in the question referenced above. https://github.com/jimeh/stub.sh/blob/master/stub.sh Haven't fully digested whether it can handle full paths and still stub the binary or not.

[grayhemp]

@dragon788 interesting tool, thank you. However, I'm a little bit afraid of stub/restore approaches. From my previous experience it's an easy way to shoot oneself in the foot.

[dragon788]

I definitely think your idea of doing it in a chroot is the safest route, that way it can be destroyed between tests to prevent side effects and can't alter the host and cause any issues.

[pszalko]

I just found this bat-mock tool and it looks very promising for unit testing bash code. I wonder if there is any progress with this issue?

[grayhemp]

@pszalko unfortunately it appeared to be harder that though and I just don't have enough time currently to address it properly. However, I would highly appreciate any PRs and/or ideas.

[dampcake]

One idea and what I have used in tests is to create a function that overrides the actual executable. Something like:

@test "my override test" {
  mock_curl="$(mock_create)"
  curl() {
    "${mock_curl}" "$@"
  }
  run something_that_call_curl
  [ "$(mock_get_call_num "${mock_curl}")" -eq 1 ]
}

In fact I had been using just regular functions to do all my mocking for quite a while until I ran into issues of not being able to get call counts or easily access multiple calls arguments.

[cescp]

Solution from @dampcake is not working for me. It seems that function defined in @test is not passed to run: my script, which calls curl, takes it from system instead of mock.

To verify it, use another non-existing command (e.g. carl): myscript.sh:

#!/usr/bin/env bash

carl --help

exit 0

and test:

@test "test mocked curl" {
    mock_curl="$(mock_create)"
    carl() {
	"${mock_curl}" "$@"
    }

    run myscript.sh
    [ "$(mock_get_call_num "${mock_curl}")" -eq 1 ]
}

When running test I get:

myscript.sh: line 3: carl: command not found

[espoelstra]

Did you create the carl script in a folder that is present in the PATH variable? Some distributions include the current directory (.) in the PATH, that has become less common since it can be a security issue, it is better to temporarily replace or extend the PATH variable explicitly for your test execution.

[cescp]

This is the complete example with curl. Both files are in the same dir. myscript.sh:

#!/usr/bin/env bash

curl

exit 0

test.bats:

#!/usr/bin/env bats

setup() {
    # https://bats-core.readthedocs.io/en/stable/
    
    # modules installed with nvm and global npm:
    # npm install -g bats-support
    # npm install -g bats-assert
    load ${NVM_BIN}/../lib/node_modules/bats-support/load.bash
    load ${NVM_BIN}/../lib/node_modules/bats-assert/load.bash

    # module installed with:
    # https://github.com/grayhemp/bats-mock#installation
    # ./build install
    load /usr/local/lib/bats-mock.bash

    # get the containing directory of this file
    # use $BATS_TEST_FILENAME instead of ${BASH_SOURCE[0]} or $0,
    # as those will point to the bats executable's location or the preprocessed file respectively
    DIR="$( cd "$( dirname "$BATS_TEST_FILENAME" )" >/dev/null 2>&1 && pwd )"
    # make executables in . visible to PATH
    PATH="$DIR:$PATH"
}


@test "test mocked curl" {
    mock_curl="$(mock_create)"
    mock_set_output ${mock_curl} "this is the output from mock"
    curl() {
	"${mock_curl}" "$@"
    }

    # run something_that_call_curl
    echo "-- running myscript.sh" >&3
    run myscript.sh
    echo "${output}" >&3
    # this check will fail
    #[ "$(mock_get_call_num "${mock_curl}")" -eq 1 ]

    # run local function
    echo "-- running local function" >&3
    run curl
    echo "${output}" >&3
    # this check will succeed
    [ "$(mock_get_call_num "${mock_curl}")" -eq 1 ]
}

Myscript is not seeing the mocked version of curl, but the system real version of it. As expected, direct run of curl function from test.bats is working fine:

$ bats test.bats
✓ test mocked curl
-- running myscript.sh
curl: try 'curl --help' or 'curl --manual' for more information
-- running local function
this is the output from mock

1 test, 0 failures

[jkenlooper]

Would using a symlink to the mocked version of curl and prepending it to the PATH inside the test work?

@test "test mocked curl" {
  mock_curl="$(mock_create)"
  mock_set_output ${mock_curl} "this is the output from mock"

  ln -s "${mock_curl}" $BATS_RUN_TMPDIR/curl
  PATH="$BATS_RUN_TMPDIR:$PATH"

  # run something_that_call_curl
  echo "-- running myscript.sh" >&3
  run myscript.sh
  echo "${output}" >&3
  # this check should pass now
  [ "$(mock_get_call_num "${mock_curl}")" -eq 1 ]

  # run local function
  echo "-- running local function" >&3
  run curl
  echo "${output}" >&3
  # this check will succeed

}

[mh182]

PR #20 contains a tentative implementation (and would also solve #17).

It doesn't use chown but it is good enough for my test setup.

Restriction to PR #20: it doesn't work for testing shell scripts with hard-coded absolute paths.