whateverable icon indicating copy to clipboard operation
whateverable copied to clipboard

Automatically detect wrong bisection results due to flapping code

Open AlexDaniel opened this issue 4 years ago • 1 comments

Currently, there's this check: https://github.com/Raku/whateverable/blob/cf9bcd03b154bcc570aca8a2628099a408cde460/xbin/Bisectable.p6#L266-L268

However, @lizmat++ and @Altai-man++ came up with an idea of doing this check automatically by comparing .moarvm files of the final commit and its parent.

I have a suspicion that rakudo in the past did not produce the same .moarvm file reproducibly, meaning that the check won't work on old commits. However, we can still do some checks, for example we can check if all changed files in the commit were to doc files.

AlexDaniel avatar Jun 27 '20 13:06 AlexDaniel

<nine> No, .moarvm files won't be identical since the compiler's id contains the SHA of the full source code

Bummer. So there's no relatively easy way of detecting that. I'll keep this open in case somebody has a better idea on how to achieve this.

Note that the commit that prompted this discussion was https://github.com/rakudo/rakudo/commit/fb6a12a, which is not just a README or doc change, so it's not very easy to figure out that it didn't really change any functionality.

IRC discussion: https://colabti.org/irclogger/irclogger_log/raku-dev?date=2020-06-27#l53

AlexDaniel avatar Jun 27 '20 13:06 AlexDaniel