cargo-bisect-rustc icon indicating copy to clipboard operation
cargo-bisect-rustc copied to clipboard

Run downloads in parallel/asynchronously

Open camelid opened this issue 5 years ago • 4 comments

I think this would make bisecting faster.

The only downside I see is that it would be more difficult to have progress bars for each of the components. Perhaps we could have one progress bar that shows the total number of bytes that have been/need to be downloaded? Then each thread/async fn would lock the progress bar and update it every 1 MB or so.

camelid avatar Oct 28 '20 05:10 camelid

I've always been of the opinion that this shouldn't make downloads faster because we should be entirely limited by network bandwidth -- that said, it would be good to confirm this (e.g., with curl or something running in parallel).

Mark-Simulacrum avatar Oct 28 '20 13:10 Mark-Simulacrum

Yeah, unfortunately my internet ends up being 4 MB/s download speed, so bisecting can take a while :/

I use --preserve, so that helps, but then I use up a ton of disk space.

camelid avatar Oct 28 '20 18:10 camelid

What we can do is parallelize compilation and downloading the next batch of artifacts. It would mean branch predicting that -- or downloading both -- but that seems not unreasonable. I guess if the compilation is fast enough it's not actually a win though.

Mark-Simulacrum avatar Oct 28 '20 20:10 Mark-Simulacrum

Re branch predicting: I once read that it can sometimes be quicker to divide the range in three using two comparisons and branches rather than dividing the range in two. The former biases the first branch to not taken instead of the pathological case of taken and not taken being equally likely.

bjorn3 avatar Oct 28 '20 21:10 bjorn3