Rémy Rakic

The compare page shows that when a benchmark stops building: e.g. in this case [the summary bot mentioned](https://github.com/rust-lang/rust/pull/95906#issuecomment-1101693631) the `rustc` benchmark stopped building and [it's showing that at the top](https://perf.rust-lang.org/compare.html?start=6fd7e9010db6be7605241c39eab7c5078ee2d5bd&end=0516711ab057d9731f55511f00f9d426bc9db724)...

> Incr comp is always disabled for non-local dependencies. Indeed I'm specifically talking about local build scripts and proc macros in that paragraph, I've added some clarification. This one is...

In light of the above feedback (thanks a bunch for that!), I have more precise data and analysis. I have more numbers [in this gist](https://gist.github.com/lqd/14c9a04a196e205c2a36ef08217af1f2) for each of the possible...

I've opened a draft PR for this: https://github.com/rust-lang/cargo/pull/10493

Maybe there can be a perf.rlo API returning the hashes for a given rollup id, as I assume they are stored in the database. cc @Mark-Simulacrum on this possibility.

An example of the common use-case tracked by this issue: an ICE appeared in a rollup and having the perf artifacts allowed manual bisection: https://github.com/rust-lang/rust/issues/101844#issuecomment-1248309182

Is the following in scope for that meeting: is it possible to ensure these optional components' tests are indeed executed on CI ? (Or is that more of a t-infra...

(I've already started looking at this and will finish this weekend)

(Note that Github duplicated comments added to existing conversations when it posted the review, so they lost the context of what they're replying to, but that's present in the original...

@rustbot author