gotestyourself
TestMain failure on re-runs seem to halt retries
It seems that if a rerun of a test fails in TestMain, then it is not subsequently retried. For example if --rerun-fails=2 and TestMain fails on the first retry, the second retry does not happen.
This happened in a private repository, so it's difficult to share the exact details. If the bug isn't obvious, I can attempt to provide a trivial repro.
A sanitized version of our logs would be something like, TestOne, TestTwo and TestThree fail in mkypkg. On retry TestOne fails in TestMain and TestTwo,TestThree pass.
FAIL Package mypkg (30.217s) # TestOne fails in TestMain
PASS mypkg/TestTwo (re-run 1) (8.46s)
PASS mypkg/TestThree (re-run 1) (8.46s)
Hello, thanks for opening this issue. This is unfortunately not a bug. --rerun-fails operates on the list of failed tests. If TestMain fails the list of failed tests isn't reported so there's no way to re-run anything. The same is true if a test panics, because all the tests may not have run.
There are other ways this could be implemented, but listing all the tests before running anything has its own set of disadvantages.
