ginkgo icon indicating copy to clipboard operation
ginkgo copied to clipboard

Unexpected output of default reporter

Open Forsworns opened this issue 3 years ago • 3 comments

After a summary of filtered specs, I got a lot of character S printed. And the number of characters is not stable. And sometime they just disappear.

$ ./test.test --ginkgo.label-filter complex

Running Suite: xxx_suite - /
================================
Random Seed: 1660010830

Will run 10 of 59 specs
SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS  # I got these `S` 
------------------------------
• [SLOW TEST] [8.830 seconds]
....

In all of 59 specs, there are 37 ones are labelled with Serial and Ordered. And other 22 are run in parallel. (Don't know if these information would help.)

I tried to read the code of default_reporter, but got no idea about this.

https://github.com/onsi/ginkgo/blob/bbefe126ba5701189f58d745e7dee6a5b44eacfc/reporters/default_reporter.go#L92

Forsworns avatar Aug 09 '22 02:08 Forsworns

can you share your code? is it open source?

onsi avatar Aug 09 '22 03:08 onsi

Sorry I can't share it.

I met this problem after upgrading ginkgo from v1 to v2. So I assumed that it's due to ginkgo and wanted to find if others met this problem.

I'll try to build a minimum example to reproduce it.

Forsworns avatar Aug 09 '22 05:08 Forsworns

@onsi I think it only indicates that some cases are skipped.

I didn't read about this from the document. A line with SSSSSSSSS is confusing. Especially when I have 49 cases skipped in the example above.

There's also a character P denoting Pending. Though with --ginkgo.vv, these confusing characters turns into the verbose one with

S [SKIPPED] [0.000 seconds] 
P [PENDING]

In fact, I think the output at the end of test is helpful enough, with

Ran 0 of 3 Specs in 0.000 seconds
SUCCESS! -- 0 Passed | 0 Failed | 0 Pending | 3 Skipped
PASS

These characters are recorded in https://github.com/onsi/ginkgo/blob/bbefe126ba5701189f58d745e7dee6a5b44eacfc/reporters/default_reporter.go#L169

Forsworns avatar Aug 09 '22 06:08 Forsworns

hey - sorry I was never able to circle back. Those Ses each represent a single skipped spec. Let me know if there's anything about this behavior I can help with. But the intention is to convey that specs are being skipped.

onsi avatar Nov 16 '22 04:11 onsi

Never mind. It would be helpful to explain it in the document :) It once made me confused with so many Ses...

Forsworns avatar Nov 16 '22 04:11 Forsworns