abi-cafe icon indicating copy to clipboard operation
abi-cafe copied to clipboard

Compile multiple tests in parallel

Open bjorn3 opened this issue 3 years ago • 4 comments

This should significantly reduce the time it takes to run all tests as most of the time is spent during compilation, which is currently single-threaded.

bjorn3 avatar Jun 28 '22 14:06 bjorn3

Main problem to solve is properly buffering up compiler messages. Right now things are fairly interactive (although CC is annoyingly prone to spitting out tons of stuff on the assumption it's running in a build.rs still).

Gankra avatar Jun 28 '22 14:06 Gankra

It seems that output buffering is already implemented for rustc. For clang and gcc using .output() should work fine too I think. For cc using a separate process for the cc crate usage would work I think.

bjorn3 avatar Jun 28 '22 14:06 bjorn3

That's probably the simplest/best approach. Main alternative is to do some upstream work to make cc-as-library-but-not-in-build.rs more usable.

Gankra avatar Jun 28 '22 14:06 Gankra

I started stubbing this out with rayon par_iter but unfortunately the current test code races on the filesystem when setting up various dirs. So this is sadly a bit non-trivial.

Gankra avatar Aug 17 '22 03:08 Gankra