TestGCRandom significantly slows running the whole test suite

[faddat]

Bug Report

Setup

Ci, per #2387

Environment: ci

node command runtime flags: ci

Config

ci

What happened?

split -d -n l/20 /home/runner/work/cometbft/cometbft/build/packages.txt /home/runner/work/cometbft/cometbft/build/packages.txt. cat /home/runner/work/cometbft/cometbft/build/packages.txt.04 | xargs go test -mod=readonly -timeout=250s -race -coverprofile=/home/runner/work/cometbft/cometbft/build/04.profile.out ok github.com/cometbft/cometbft/internal/bits 1.030s coverage: 85.8% of statements ok github.com/cometbft/cometbft/internal/blocksync 30.600s coverage: 77.9% of statements coverage: 57.9% of statements panic: test timed out after 4m10s running tests: TestGCRandom (1m58s)

goroutine 21 [running]: testing.(*M).startAlarm.func1() /opt/hostedtoolcache/go/1.21.8/x64/src/testing/testing.go:2259 +0x259 created by time.goFunc /opt/hostedtoolcache/go/1.21.8/x64/src/time/sleep.go:176 +0x45

goroutine 1 [chan receive, 1 minutes]: testing.(*T).Run(0xc00010cd00, {0x71e1ee, 0xc}, 0x7470e0) /opt/hostedtoolcache/go/1.21.8/x64/src/testing/testing.go:1649 +0x871 testing.runTests.func1(0x0?) /opt/hostedtoolcache/go/1.21.8/x64/src/testing/testing.go:2054 +0x85 testing.tRunner(0xc00010cd00, 0xc000187ae8) /opt/hostedtoolcache/go/1.21.8/x64/src/testing/testing.go:1595 +0x262 testing.runTests(0xc000120be0?, {0x90ad60, 0x6, 0x6}, {0x1c?, 0x1e?, 0x910e20?}) /opt/hostedtoolcache/go/1.21.8/x64/src/testing/testing.go:2052 +0x8ae testing.(*M).Run(0xc000120be0) /opt/hostedtoolcache/go/1.21.8/x64/src/testing/testing.go:1925 +0xcd8 main.main() _testmain.go:97 +0x2e5

goroutine 4 [semacquire]: sync.runtime_Semacquire(0xc0161ee058?) /opt/hostedtoolcache/go/1.21.8/x64/src/runtime/sema.go:62 +0x25 sync.(*WaitGroup).Wait(0xc0161ee050) /opt/hostedtoolcache/go/1.21.8/x64/src/sync/waitgroup.go:116 +0xa5 github.com/cometbft/cometbft/internal/clist.TestGCRandom(0xc000082b60) /home/runner/work/cometbft/cometbft/internal/clist/clist_test.go:150 +0x40b testing.tRunner(0xc000082b60, 0x7470e0) /opt/hostedtoolcache/go/1.21.8/x64/src/testing/testing.go:1595 +0x262 created by testing.(*T).Run in goroutine 1 /opt/hostedtoolcache/go/1.21.8/x64/src/testing/testing.go:1648 +0x846 FAIL github.com/cometbft/cometbft/internal/clist 250.218s ok github.com/cometbft/cometbft/internal/cmap 1.014s coverage: 91.2% of statements FAIL

What did you expect to happen?

This garbage collector test shouldn't approximately double the time it takes to run all of the tests in comet.

How to reproduce it

Tell CI that it has 210 seconds to run all of the tests by changing the value in Makefile. This test (and no others) will then fail.

Logs

dump_consensus_state output

Anything else we need to know