whateverable
whateverable copied to clipboard
Automatically detect wrong bisection results due to flapping code
Currently, there's this check: https://github.com/Raku/whateverable/blob/cf9bcd03b154bcc570aca8a2628099a408cde460/xbin/Bisectable.p6#L266-L268
However, @lizmat++ and @Altai-man++ came up with an idea of doing this check automatically by comparing .moarvm
files of the final commit and its parent.
I have a suspicion that rakudo in the past did not produce the same .moarvm file reproducibly, meaning that the check won't work on old commits. However, we can still do some checks, for example we can check if all changed files in the commit were to doc files.
<nine> No, .moarvm files won't be identical since the compiler's id contains the SHA of the full source code
Bummer. So there's no relatively easy way of detecting that. I'll keep this open in case somebody has a better idea on how to achieve this.
Note that the commit that prompted this discussion was https://github.com/rakudo/rakudo/commit/fb6a12a, which is not just a README or doc change, so it's not very easy to figure out that it didn't really change any functionality.
IRC discussion: https://colabti.org/irclogger/irclogger_log/raku-dev?date=2020-06-27#l53