Backticks in analysis test string literals are evaluated as shell script

[dgp1130]

trafficstars

I have a minimal reproduction repo here: https://github.com/dgp1130/skylib-test-eval. (I'm using Linux. Windows may or may not reproduce.)

TL;DR: Using analysistest to assert on an error message containing backticks will evaluate the text in those backticks in the shell.

An innocuous assertion like:

asserts.expect_failure(env, "Oh noes! Value `foo` should have been `bar`!")

Will output failure messages like:

==================== Test output for //:failure_testing_test:
/home/douglasparker/.cache/bazel/_bazel_douglasparker/f8daf9e1b24fe39f31e93c50d2583d6b/sandbox/linux-sandbox/9/execroot/__main__/bazel-out/k8-fastbuild/bin/failure_testing_test.sh.runfiles/__main__/failure_testing_test.sh: line 1: foo: command not found
/home/douglasparker/.cache/bazel/_bazel_douglasparker/f8daf9e1b24fe39f31e93c50d2583d6b/sandbox/linux-sandbox/9/execroot/__main__/bazel-out/k8-fastbuild/bin/failure_testing_test.sh.runfiles/__main__/failure_testing_test.sh: line 1: bar: command not found
/home/douglasparker/.cache/bazel/_bazel_douglasparker/f8daf9e1b24fe39f31e93c50d2583d6b/sandbox/linux-sandbox/9/execroot/__main__/bazel-out/k8-fastbuild/bin/failure_testing_test.sh.runfiles/__main__/failure_testing_test.sh: line 1: bar: command not found
/home/douglasparker/.cache/bazel/_bazel_douglasparker/f8daf9e1b24fe39f31e93c50d2583d6b/sandbox/linux-sandbox/9/execroot/__main__/bazel-out/k8-fastbuild/bin/failure_testing_test.sh.runfiles/__main__/failure_testing_test.sh: line 1: foo: command not found
In test _failure_testing_test_impl from //:build_defs_test.bzl: Expected errors to contain 'Oh noes! Value  should have been !' but did not. Actual errors:
Traceback (most recent call last):
	File "/home/douglasparker/Source/bazel-test-eval/BUILD", line 3
		_my_rule(name = 'this_should_fail')
	File "/home/douglasparker/Source/bazel-test-eval/build_defs_test.bzl", line 15, in _my_rule_impl
		fails_with_backtick_in_message()
	File "/home/douglasparker/Source/bazel-test-eval/build_defs.bzl", line 2, in fails_with_backtick_in_message
		fail(<1 more arguments>)
Oh noes! Value  should have been !

It appears that foo and bar are being treated as a subshell and evaluated inline. I believe the evaluation is coming from here. Running one of the generated shell scripts reproduces the problem:

$ bazel-bin/failure_testing_test.sh.runfiles/__main__/failure_testing_test.sh
bazel-bin/failure_testing_test.sh.runfiles/__main__/failure_testing_test.sh: line 1: foo: command not found
bazel-bin/failure_testing_test.sh.runfiles/__main__/failure_testing_test.sh: line 1: bar: command not found
bazel-bin/failure_testing_test.sh.runfiles/__main__/failure_testing_test.sh: line 1: bar: command not found
bazel-bin/failure_testing_test.sh.runfiles/__main__/failure_testing_test.sh: line 1: foo: command not found
In test _failure_testing_test_impl from //:build_defs_test.bzl: Expected errors to contain 'Oh noes! Value  should have been !' but did not. Actual errors:
Traceback (most recent call last):
        File "/home/douglasparker/Source/bazel-test-eval/BUILD", line 3
                _my_rule(name = 'this_should_fail')
        File "/home/douglasparker/Source/bazel-test-eval/build_defs_test.bzl", line 15, in _my_rule_impl
                fails_with_backtick_in_message()
        File "/home/douglasparker/Source/bazel-test-eval/build_defs.bzl", line 2, in fails_with_backtick_in_message
                fail(<1 more arguments>)
Oh noes! Value  should have been !
$ cat bazel-bin/failure_testing_test.sh.runfiles/__main__/failure_testing_test.sh
cat << EOF
In test _failure_testing_test_impl from //:build_defs_test.bzl: Expected errors to contain 'Oh noes! Value `foo` should have been `bar`!' but did not. Actual errors:
Traceback (most recent call last):
        File "/home/douglasparker/Source/bazel-test-eval/BUILD", line 3
                _my_rule(name = 'this_should_fail')
        File "/home/douglasparker/Source/bazel-test-eval/build_defs_test.bzl", line 15, in _my_rule_impl
                fails_with_backtick_in_message()
        File "/home/douglasparker/Source/bazel-test-eval/build_defs.bzl", line 2, in fails_with_backtick_in_message
                fail(<1 more arguments>)
Oh noes! Value `bar` should have been `foo`!

EOF
exit 1