Allow using prebuilt sanitizer libraries

[jakos-sec]

Proposal

In the efforts of the Rust goal Rust Stabilization of MemorySanitizer and ThreadSanitizer Support, we are looking for a solution to distribute prebuilt sanitizer instrumented standard libraries. This was requested by previous reviews, in order to stabilize sanitizers that require all code to be instrumented since otherwise they generate false positives (like the MemorySanitizer and ThreadSanitizer). Currently the only way to realistically use those is with -Z build-std, but if we stabilize the sanitizers, it is odd to still be dependent on another unstable flag.

We should therefore have an easier way to use stabilized sanitizers with sanitizer instrumented standard libraries without build-std by providing the instrumented standard library artifacts through e.g. rustup. I've originally approached the infra team but was directed to first addressing the necessary compiler changes before looking into distributing the artifacts (#t-infra/bootstrap > 2025H2 Goal Review @ 💬).

Together with some of the code that eholk provided me, I made a prototype in a local branch with 2 commits.

• Use lib-asan sysroot when ASAN is enabled with the compiler change
• Generate ASAN-enabled sysroot with a protoype of the bootstrap changes to actually test the compiler changes

The high-level design proposal looks like this:

Currently rustc has a single --sysroot that allows to set the entire sysroot but there is no way to toggle between different standard library variants there.

We could have a structure inside that looks something like this:

.../lib/rustlib/
└── x86_64-unknown-linux-gnu/
    ├── lib/          # The default standard library
    ├── lib-asan/     # The variant built with AddressSanitizer
    └── lib-msan/     # The variant built with MemorySanitizer

In order to do so, we would need: A) compiler support to enable the variant required and B) bootstrap support in order to provide the variants through rustup.

In order to allow build-std to use -Zsanitizer=address without using the prebuilt artifacts, I introduced the rustc flag -Zsanitizer-use-prebuilt-library, which once the artifacts are shipped should default to yes and set to no by cargo when using build-std.

Some more context can also be found here.

Mentors or Reviewers

Mentor/Reviewer: rcvalle

Process

The main points of the Major Change Process are as follows:

• [x] File an issue describing the proposal.
• [ ] A compiler team member who is knowledgeable in the area can second by writing @rustbot second or kickoff a team FCP with @rfcbot fcp $RESOLUTION.
  • Refer to Proposals, Approvals and Stabilization docs for when a second is sufficient, or when a full team FCP is required.
• [ ] Once an MCP is seconded, the Final Comment Period begins.
  • Final Comment Period lasts for 10 days after all outstanding concerns are solved.
  • Outstanding concerns will block the Final Comment Period from finishing. Once all concerns are resolved, the 10 day countdown is restarted.
  • If no concerns are raised after 10 days since the resolution of the last outstanding concern, the MCP is considered approved.

You can read more about Major Change Proposals on forge.

[!CAUTION]

Concerns (2 active)

• why-not-third-party-tool
• what-about-combinatorics

Managed by @rustbot—see help for details.

[rustbot]

[!IMPORTANT] This issue is not meant to be used for technical discussion. There is a Zulip stream for that. Use this issue to leave procedural comments, such as volunteering to review, indicating that you second the proposal (or third, etc), or raising a concern that you would like to be addressed.

Concerns or objections can formally be registered here by adding a comment.

@rustbot concern reason-for-concern
<description of the concern>

Concerns can be lifted with:

@rustbot resolve reason-for-concern

See documentation at https://forge.rust-lang.org

cc @rust-lang/compiler

[workingjubilee]

@rustbot concern why-not-third-party-tool

During discussion, @Kobzol came up with the note that a 3rd party tool (specifically, a cargo subcommand) could do better at assembling a smaller set of "legos" (build flags, paths, etc.) as an intermediate state. My form of this concern is as follows:

• this adds magic behavior, and then adds flags just to disable a newer, more "magic" behavior, which seems worse than just using --sysroot or similar... but a tool that integrates with cargo (or possibly just cargo, if you can pitch them on it?) and takes care of handling --sysroot seems like it can provide good UX for this.
• this change makes implementing -Zbuild-std ever-so-slightly harder, and even adding an ankle-high fence in front of -Zbuild-std seems bad because of how complex it is already and how many non-Cargo build systems will want/need to reimplement it.
• at the same time, -Zbuild-std can provide surpassingly better support for using sanitizers, at much less cost to our CI and release, so it's unclear it is desirable to invest effort on maintaining this on the compiler side.

[workingjubilee]

@rustbot concern what-about-combinatorics

Doesn't this design leave the use-case of wanting to compile code with multiple sanitizers enabled fundamentally unaddressed? Yes, it's hard to address the combinatorics that invokes without just rebuilding std "on-demand", but that seems like it points towards this being the wrong solution as a "just for now" alternative.

[jakos-sec]

Summarizing the follow up discussion from zulip:

regarding why-not-third-party-tool: The most favored solution to the problem seemed to instead of adding a new rustc flag, to add new targets that simply have sanitizers enabled by default (I've just opened a new MCP to allow such targets as Tier 2 to distribute the standard libraries for it: https://github.com/rust-lang/compiler-team/issues/951)

regarding what-about-combinatorics: Combining sanitizers is not that common, many are incompatible with each other (pretty much all that use shadow memory which most of them) and even for the combinations left, I'm not sure how many actually use them so I think it's fine to say you need to use -Zbuild-std if you need that special case since that remains possible, you'll just not get prebuilt artifacts that were instrumented with both.