clib
clib copied to clipboard
Randomly failing tests on MacOS
Taking a look at the actions results it seems some tests fail on Mac, but not always.
Running clib(1) tests
./test/gh-35-exit-codes.sh: line 8: 21111 Segmentation fault: 11 clib install stephenmathieson/rot132.c > /dev/null 2>&1
(✓) test/gh-35-exit-codes.sh
./test/help.sh: line 3: 21113 Segmentation fault: 11 clib help 2> /dev/null
Expected `clib help` to fail
(✖) test/help.sh
./test/install-binary-dependencies.sh: line 6: 21115 Segmentation fault: 11 clib install -c stephenmathieson/[email protected] -P tmp > /dev/null
Failed to install stephenmathieson/tabs-to-spaces
(✖) test/install-binary-dependencies.sh
./test/install-brace-expansion.sh: line 9: 21117 Segmentation fault: 11 clib install -c -o tmp stephenmathieson/trim.c stephenmathieson/case.c > /dev/null
expecting successful exit code
(✖) test/install-brace-expansion.sh
./test/install-deps-from-package-json.sh: line 23: 21122 Segmentation fault: 11 clib install > /dev/null 2>&1
expecting exit code of 1
(✖) test/install-deps-from-package-json.sh
./test/install-multiple-clibs-libs.sh: line 9: 21124 Segmentation fault: 11 clib install -c -o tmp ms file hash > /dev/null
expecting successful exit code
(✖) test/install-multiple-clibs-libs.sh
./test/install-multiple-libs.sh: line 10: 21126 Segmentation fault: 11 clib install -c -o tmp stephenmathieson/case.c stephenmathieson/trim.c > /dev/null
expecting successful exit code
(✖) test/install-multiple-libs.sh
Edit: I can't reproduce any of it on Linux now.
This log file might give some hints. It's from when I built with gcc 11.1.0 on Debian Bullseye and added the -fanalyzer
flag.
I had some problems recently with tests randomly failing on MacOS using Travis CI. Turned out to be a cache issue.
Btw, I've found Coverity to be a useful tool for testing. Although it sometimes gives false positives, it's pretty good at spotting potential resource leaks or buffer overflows.