Shell tests in rules_scala don't use the Bazel sh_test rule

[Wyverald]

trafficstars

Shell tests in rules_scala are run by executing a huge shell script (//:test_rules_scala.sh) which runs each individual shell test function sequentially. This is very painful for developers:

• No caching: You might run the whole test suite and realize a test case near the end fails. After you fix it, you'd need to rerun all test cases before it to reach it again.
• Eager failure: You might run through 50 test cases and reach the 51st case which fails. After you fix it, you'd need run the suite again to discover that the 52nd case also fails.

Converting these shell tests to use Bazel's sh_test rule would save a lot of headache.

[Wyverald]

Another point -- if you run some tests like test_scala_import_expect_failure_on_missing_direct_deps_warn_mode twice in a row, the second run would fail. You need a bazel clean in between. This violates hermeticity.

[liucijus]

@Wyverald these tests need to test Bazel failure scenarios, and/or pass different arguments. I don't think there is a way to run them from sh_test without running Bazel inside Bazel.

[Wyverald]

You can in fact run Bazel inside Bazel -- this is what Bazel's own test suite does (example: https://cs.opensource.google/bazel/bazel/+/master:src/test/shell/bazel/bazel_workspaces_test.sh;l=1?q=bazel_workspace&sq=&ss=bazel%2Fbazel)