rust-lang
Rustup Pull-Through Cache
Problem you are trying to solve
The Internet seems to be decomposing and Open Source project download channels are collapsing in the process. Rustup is not immune unfortunately and we're seeing a significant increase in CI failures and hearing from devs across various client environments which rely on rustup mostly due to:
error: failed to download file error=Reqwest(reqwest::Error { kind: Request, url: "https://static.rust-lang.org/dist/2025-06-26/rustc-1.88.0-x86_64-unknown-linux-gnu.tar.xz", source: TimedOut })
info: retrying download for 'https://static.rust-lang.org/dist/2025-06-26/rustc-1.88.0-x86_64-unknown-linux-gnu.tar.xz'
error: failed to download file error=Reqwest(reqwest::Error { kind: Request, url: "https://static.rust-lang.org/dist/2025-06-26/rustc-1.88.0-x86_64-unknown-linux-gnu.tar.xz", source: TimedOut })
info: retrying download for 'https://static.rust-lang.org/dist/2025-06-26/rustc-1.88.0-x86_64-unknown-linux-gnu.tar.xz'
error: failed to download file error=Reqwest(reqwest::Error { kind: Request, url: "https://static.rust-lang.org/dist/2025-06-26/rustc-1.88.0-x86_64-unknown-linux-gnu.tar.xz", source: TimedOut })
info: retrying download for 'https://static.rust-lang.org/dist/2025-06-26/rustc-1.88.0-x86_64-unknown-linux-gnu.tar.xz'
error: failed to download file error=Reqwest(reqwest::Error { kind: Request, url: "https://static.rust-lang.org/dist/2025-06-26/rustc-1.88.0-x86_64-unknown-linux-gnu.tar.xz", source: TimedOut }
Not sure if upstream has rate limiting or if intermediate providers are unhappy with the IO patterns and blocking folks (we're seeing this at various sites in the US and EU).
Solution you'd like
Proposing a rustup proxy serve command which would allow an instance of the binary to serve as a caching proxy for other instances and use of an env var or cli flag to direct instances of rustup performing the installation to get service from the proxy forcing a pull-through on new content and pull-back for cached content.
Should help reduce the load on public servers considerably if CIs were to leverage an innate function for the binary for this instead of having to wire up their own solutions (or just not bother and maul public servers with bandwidth consuming requests).
Notes
For some context: we're a security outfit (predating Rust's existence), manage risk for organizations far larger than the original parent co.
Consolidation of infrastructure carries risk profiles to the ecosystem to include
- being blocked by regimes and providers
- targeted for "lawful takeover" by hostile entities to use your resource platform against their adversaries
- scaled down by intermediate parties (QoS) and MiTMd
- provide centralized points for denial of service attacks
- the fun stuff we're seeing over at RubyGems which smells a bit like the fall of freenode but likely bad for the Ruby community in any case.
Enabling users of the ecosystem to be the ecosystem diffuses the risk profile and decentralizes some of cost borne by the organization hosting all of these downloads. Natively supported cache tiering and third-party mirroring, P2P distribution, and the attestation + validation mechanisms which are required to ensure safety of such approaches would cost less than the bandwidth and DoS protection budget today and far less than the PR nightmare of cleaning up any of the concerns listed above (unless the entity is owned by NK through a proxy shell co at that point) will cost when they hit the fan.
Some recent context:
- #4440
- #4218
- #4439
I think this means the next release will already substantially improve the situation.
A system or mirroring sounds nice but given that rustup today consists of an all-volunteer team, I don't think such features will just materialize. If you or your clients are serious about funding such an effort, happy to talk about that.
Funny you ask @djc, we (Semper Victus) just picked up funding the revival and architectural overhaul of Rubinius so our FOSS budget's a bit tight. That said, we do have talent/tooling and wiring up some of the guts involved and a PoC shouldn't be too hard (or in "consultancy terms," expensive).
Going to tag in @dmaynor to dig in and either address or dispatch to team as needed. Apologies, i generally love this sort of stuff but my role these days doesn't leave much time to scalpel-around.
Appreciate your funding of other OSS efforts! I'm not even sure if such an effort should be part of rustup, but there might be parts that can be reused.
@djc - to the question of whether a function like this should be in the toolchain: what if any current mitigations exist for loss of the domain/servers by rust-lang org?
Example threat model: Open Source is not immune from polarizing sociopolitical change and every large FOSS org has at least one or two people who have made public statements in the past running afoul of some ideology or another - statements likely to surface in the future given the increased scrutiny and scanning capabilities being employed by authorities in determining who's on which side of the societal schisms afoot. Domain seizures are common fare when dealing with criminal or terrorist organizations and any donations being made to such entities could expose the donors to criminal liability as well. In a centralized model, entities without appropriate leverage and connections of their own running afoul of people in power even accidentally could see themselves wiped out overnight.
To my understanding there is nothing like .net/java support contracts for rust while trillions of dollars of commerce are done on the language; hence the thought process here is to de-risk the ecosystem a bit for individuals and commercial/government entities by decreasing direct dependence on a single vector of access.
I said "part of rustup", not "in the toolchain". I think discussions on the risks you mention won't be effective in the rustup issue tracker. For organizations that worry about these kinds of risk, I'd suggest becoming a Rust Foundation member and discussing the issue in that venue.