JuliaLang
Tests on CI should use multiple threads
We have bugs involving multi-threading (#963, #990) and should add regressions tests.
However, if I understand this correctly, they could even give a false sense of reliability, as CI runs PackageCompiler.jl single-threaded and so they would appear to succeed even if they would fail in a multi-threaded context. We could run processes we create from PackageCompiler.jl multithreaded, but I think the most realistic use case is to just set JULIA_NUM_THREADS for CI. This way everything in PackageCompiler.jl is automatically tested in a multi-threaded context.
It's not uncommon to launch subprocesses with different env and args from runtests.jl. Here's one example: https://github.com/EnzymeAD/Enzyme.jl/blob/cbadb626de5f0d7c011356933422133ef9fdebd4/test/runtests.jl#L1634-L1639.
The benefit of this approach is that the different launch configurations explicitly become part of the test suite, so they won't inadvertently get lost when, say, the package moves to a different CI provider. They also become part of PkgEval runs.
But the best solution is whatever actually gets implemented. I don't use this package myself, just chiming in from the Discourse threads, so I won't put my name down for one or the other.
