Tests relying on `reqwest::Client` may fail with "dispatch task is gone: runtime dropped the dispatch task"

[rami3l]

Verification

• [X] I searched for recent similar issues at https://github.com/rust-lang/rustup/issues?q=is%3Aissue+is%3Aopen%2Cclosed and found no duplicates.
• [X] I am on the latest version of Rustup according to https://github.com/rust-lang/rustup/tags and am still able to reproduce my issue.

Problem

This is causing irregular failures for different CI platforms:

• Linux
  • https://github.com/rust-lang/rustup/actions/runs/9303236385/job/25605096671
  • https://github.com/rust-lang/rustup/actions/runs/9354772182/job/25748366479
  • https://github.com/rust-lang/rustup/actions/runs/9337787790/job/25700115067
  • https://github.com/rust-lang/rustup/actions/runs/9343224087/job/25712479916
• Windows
  • https://github.com/rust-lang/rustup/actions/runs/9335696949/job/25695378538

The error message here is identical (although the exact test that failed might differ):

running 395 tests
...
test suite::cli_exact::check_updates_with_update ... FAILED
error: could not download file from 'https://static.rust-lang.org/rustup/release-stable.toml' to '/tmp/rustup-updateD58GcC/release-stable.toml': failed to make network request: error sending request for url (https://static.rust-lang.org/rustup/release-stable.toml): client error (SendRequest): dispatch task is gone: runtime dropped the dispatch task

Notes

Possibly related to https://github.com/hyperium/hyper/issues/2312, https://github.com/Azure/azure-sdk-for-rust/pull/1550 or https://github.com/fussybeaver/bollard/issues/387#issuecomment-2008133973.

I'm pretty sure that #3367 has caused this.

[rami3l]

According to the error message format and my IDE searching the current codebase, this is a RustupError::DownloadingFile. This kind error message can only be emitted from here:

https://github.com/rust-lang/rustup/blob/6d4376180d10e35daec5453546e508eb4726fa52/src/utils/utils.rs#L154-L190

[twuebi]

Hi @rami3l, I also encountered the same issue with tests, searching for the cause, I came across this thread on the rust-lang forums. We started encountering the problem over at lakekeeper (https://github.com/lakekeeper/lakekeeper/issues/548) after making our clients static with lazy-locks like you are doing here, I think that is the cause for the failures you've been observing.

[rami3l]

Hi @rami3l, I also encountered the same issue with tests, searching for the cause, I came across this thread on the rust-lang forums. We started encountering the problem over at lakekeeper (lakekeeper/lakekeeper#548) after making our clients static with lazy-locks like you are doing here, I think that is the cause for the failures you've been observing.

@twuebi Thanks a lot for the info! I think your observation is likely relevant as we currently use in-process testing for some commands even if it appears to be out-of-process (https://github.com/rust-lang/rustup/pull/3891).

It could indeed be possible that something like https://github.com/rust-lang/rustup/pull/3891/commits/2261e266da6b8e6218757aaa857c57c04ae29d78 is required for reqwest::Client as well.

Anyway, I'll find some time to look into this.