gotestsum icon indicating copy to clipboard operation
gotestsum copied to clipboard

gotestsum reports nonexistent errors

Open fgm opened this issue 1 year ago • 3 comments

Here is the situation: tests succeed with go test, both on their own and in the embedded go test -json command for gotestsum, but when gotestsum is done, it reports one error where none exists; and what's more it returns with status 0.

# Testing with gotestsum
./bin/gotestsum -f standard-verbose -- -v -count=1 -race ./somepackage
# github.com/someorg/somepackage/some.test
=== RUN   TestFoo
=== RUN   TestFoo/bar
--- PASS: TestFoo (0.00s)
    --- PASS: TestFoo/bar (0.00s)
PASS
ok  	github.com/someorg/somepackage	2.586s

=== Errors

DONE 2 tests, 1 error in 4.981s

$ echo $?
0

# Testing with go test
$ go test                              -v -count=1 -race ./somepackage 
# github.com/someorg/somepackage/some.test
=== RUN   TestFoo
=== RUN   TestFoo/bar
--- PASS: TestFoo (0.00s)
    --- PASS: TestFoo/bar (0.00s)
PASS
ok  	github.com/someorg/somepackage	1.636s

$ echo $?
0

Tested on

  • macOS Sonoma, Apple Silicon.
  • Both with go 1.22.6 and 1.23.0.
  • gotestsum version dev

fgm avatar Aug 22 '24 11:08 fgm

Hello, thank you for the bug report! Errors come from any stderr that is emitted from the test binary. I don't see any lines in this example. Do you have some code I can run to reproduce?

dnephin avatar Aug 29 '24 02:08 dnephin

I have a similar problem. Even the junit.xml starts with

<?xml version="1.0" encoding="UTF-8"?>
<testsuites tests="6412" failures="0" errors="1" time="412.795940">

but I literally can't find any error, failure, stack, NOTHING that could have gone wrong in the logs, or the report.

ikari-pl avatar Apr 05 '25 09:04 ikari-pl

@ikari-pl what version of gotestsum ? Does it happen every time, or occasionally?

This can happen when go test -json writes something to stderr. In this case it seems like it's writing an empty line, which is strange.

I guess we can fix this by adding something here that ignores blank lines.

if strings.TrimSpace(line) == "" {
    continue
}

If someone wants to submit a PR, I can approve it.

dnephin avatar May 06 '25 03:05 dnephin