google
[BUG] `user_counters_tabular_test` is failing
Hi!
I forked this repository and after one of the syncs one action failed on windows. I tried recreating it on Ubuntu 22.04 and was successful. I used debug build.
gdb -ex=r --args ./test/Debug/user_counters_tabular_test:
BM_CounterSet2_Tabular/threads:16 9.18 ns 6.05 ns 121737072 Bat=30 Baz=40 Foo=10
/home/senyai/projects/extra/benchmark/test/output_test_helper.cc:118: CheckCase: Check `TC.match_rule != MR_Next' failed. Expected line "Benchmark Time CPU Iterations UserCounters..." to match regex "^Benchmark %s Time %s CPU %s Iterations %s Bar %s Bat %s Baz %s Foo %s Frob %s Lob$"
actual regex string "^Benchmark[ ]+Time[ ]+CPU[ ]+Iterations[ ]+Bar[ ]+Bat[ ]+Baz[ ]+Foo[ ]+Frob[ ]+Lob$"
started matching near: Benchmark Time CPU Iterations UserCounters...
Thread 1 "user_counters_t" received signal SIGABRT, Aborted.
__pthread_kill_implementation (no_tid=0, signo=6, threadid=140737352611648) at ./nptl/pthread_kill.c:44
44 ./nptl/pthread_kill.c: No such file or directory.
(gdb) bt
#0 __pthread_kill_implementation (no_tid=0, signo=6, threadid=140737352611648) at ./nptl/pthread_kill.c:44
#1 __pthread_kill_internal (signo=6, threadid=140737352611648) at ./nptl/pthread_kill.c:78
#2 __GI___pthread_kill (threadid=140737352611648, signo=signo@entry=6) at ./nptl/pthread_kill.c:89
#3 0x00007ffff7842476 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
#4 0x00007ffff78287f3 in __GI_abort () at ./stdlib/abort.c:79
#5 0x0000555555575323 in benchmark::internal::CallAbortHandler () at /home/senyai/projects/extra/benchmark/test/../src/check.h:39
#6 0x000055555557547a in benchmark::internal::CheckHandler::~CheckHandler (this=0x7fffffffcf28, __in_chrg=<optimized out>)
at /home/senyai/projects/extra/benchmark/test/../src/check.h:65
#7 0x000055555557af41 in internal::(anonymous namespace)::CheckCase (remaining_output=..., TC=...,
not_checks=std::vector of length 0, capacity 0) at /home/senyai/projects/extra/benchmark/test/output_test_helper.cc:118
#8 0x000055555557b26f in internal::(anonymous namespace)::CheckCases (checks=std::vector of length 41, capacity 41 = {...}, output=...)
at /home/senyai/projects/extra/benchmark/test/output_test_helper.cc:139
#9 0x000055555557db5e in RunOutputTests (argc=1, argv=0x7fffffffdda8) at /home/senyai/projects/extra/benchmark/test/output_test_helper.cc:465
#10 0x0000555555565853 in main (argc=1, argv=0x7fffffffdda8) at /home/senyai/projects/extra/benchmark/test/user_counters_tabular_test.cc:565
(gdb) q
Oops, I this is a popular one. Mentioned in #1279 and #1665. Maybe it's time I have a look at it!
IIRC the measured time for some previous benchmark happens to be 0, and the regex matches something else.
this has been an ongoing issue with the windows benchmarks over years. i never got to the bottom of why it happens, whether it's a legitimate result (zomg windows so fast) or a bad timer implementation...
If we want, we might be able to ditch windows runners altogether and do cross-compilation from linux through wine. Though better yet, tests should not actually be using real timings, they should all use manual times. At least i would think so.
this has been an ongoing issue with the windows benchmarks over years. i never got to the bottom of why it happens, whether it's a legitimate result (zomg windows so fast) or a bad timer implementation...
:) It would most certainy be timing granularity, and rounding it down and not up.
mocking the timer results sounds like a great idea. i'm not sure the project is well structured to allow it but maybe some judicious gmock usage could do the trick.
IIRC the measured time for some previous benchmark happens to be 0, and the regex matches something else.
All tests are run with --benchmark_min_time=0.01s. Isn't this supposed to address the issue? I can fix the issue by adding "inf" to the regex.