Make it easier to run for contributors

[AlexDaniel]

Currently you can only test some functions of Unicodable, other bots require a bunch existing rakudo revisions (which takes a few days to get done). In other words, you can only develop stuff on the server where the bots are running…

We should add some “contributor” mode for build.p6 that would only build maybe 5 revisions, and make sure that most bots are functional with just these 5 builds.

Bots that you can run and TEST locally:

• [x] Bisectable – OK but may fail on the first run
• [x] Committable – OK but may fail on the first run
• [x] Benchable – OK but may fail on the first run
• [x] Evalable – OK
• [ ] Statisfiable
• [x] Unicodable – OK
• [ ] Bloatable – requires bloaty that you have to compile yourself…
• [ ] Quotable
• [ ] Greppable – you have to run it twice to make it pass its tests (it pulls perl6-all-modules on the first run and times out), also seems to be too slow on my machine to get things done in time
• [ ] Coverable – failing tests
• [ ] Releasable
• [ ] Nativecallable
• [ ] Squashable – no tests yet
• [ ] Reportable – ± no tests yet, but will run fine locally (more info on Reportable#running-the-bot-locally)
• [x] Notable
• [ ] Shareable – no tests yet

[AlexDaniel]

Recently there was some work in that direction: https://github.com/perl6/whateverable/commit/2c23d3edaf0be1ae00cc7f44876983574a78faf8, https://github.com/perl6/whateverable/commit/7c0649d325ced431f8dd20ae9f3fe3a40dcd388e

[AlexDaniel]

Actually, almost every bot is now almost 100% usable on linux x86_64 (it will pull builds from the server and you don't have to wait for anything). That's pretty cool, and someone just has to confirm that every bot is indeed usable.

Alternatively we can try running through the test suite locally, but that would probably time out for many tests because it takes a second or two to download the required build.

So, we're almost there, just a bit more work and we'll be able to close this.

[AlexDaniel]

Ah, there's also a problem with rakudo/rakudo#1501, but this shouldn't affect most of the bots (just bisectable I think).

[AlexDaniel]

Not only you can run all bots locally now, but I just pushed a set of commits that make it easy to run tests yourself! It's a bit early to say that this issue is fully resolved as I haven't tested every bot yet (some of them are segfaulting anyway :cry:), but at least one bot (unicodable) is passing its tests fully.

[AlexDaniel]

• [x] It should complain if zstd or lrzip is not installed.

[AlexDaniel]

• [x] Committable is trying to fetch a lot of unrelated stuff but otherwise it works. Example:

ok 126 - Did you mean some commit?
Attempting to fetch 1049b41c5835d4c873b2e8772ba5e066df52cbd1…
Attempting to fetch 1049b41c5835d4c873b2e8772ba5e066df52cbd1…
Attempting to fetch 10eba0168e4cf8ed460a83eb6c40c0c2052bf7fa…
Attempting to fetch 10eba0168e4cf8ed460a83eb6c40c0c2052bf7fa…
Attempting to fetch 455467b8e3370396811394a7a9032cfe3bc38f34…
Attempting to fetch 455467b8e3370396811394a7a9032cfe3bc38f34…
Attempting to fetch f86a1a83111ad0c78e5ecd0aa1775d2b293430a4…
Attempting to fetch f86a1a83111ad0c78e5ecd0aa1775d2b293430a4…
Attempting to fetch f024a60f54745d5c4f9d52f896e00907436e7daf…
Attempting to fetch f024a60f54745d5c4f9d52f896e00907436e7daf…
Attempting to fetch b15bdcb315c86c472c267b3d204e4a384398d881…
Attempting to fetch b15bdcb315c86c472c267b3d204e4a384398d881…
Attempting to fetch c37bc4916ff13bde888a66271b13cd9039a05733…
Attempting to fetch c37bc4916ff13bde888a66271b13cd9039a05733…
Attempting to fetch 9743c6bd7a96c144505640cf16c4f0441cb946e1…
Attempting to fetch 9743c6bd7a96c144505640cf16c4f0441cb946e1…
Attempting to fetch 88221cff0c1199c1220d88f80c711c868fc9e198…
Attempting to fetch 88221cff0c1199c1220d88f80c711c868fc9e198…
Attempting to fetch 6d7c9cc61cf1c89d438ba30bb62fbbff6f1bff4f…
Attempting to fetch 6d7c9cc61cf1c89d438ba30bb62fbbff6f1bff4f…
Attempting to fetch 952d407a0b455fc5757a23d93de332fb2b9a1b89…
Attempting to fetch 42f8ead6ed70a65aefc770ddd76bdbdbcf449bcd…
Attempting to fetch 42f8ead6ed70a65aefc770ddd76bdbdbcf449bcd…
Attempting to fetch 3a8d48a50bbaf28cb5ec6aa8a381d72db63da1a2…
Attempting to fetch 3a8d48a50bbaf28cb5ec6aa8a381d72db63da1a2…
Attempting to fetch 55dbc07f67d413063db2b93bda8c182ac04b2fe2…
Attempting to fetch 55dbc07f67d413063db2b93bda8c182ac04b2fe2…
Attempting to fetch 231470f872b5d7be21c45657e04b20c7db2af6fd…
Attempting to fetch 231470f872b5d7be21c45657e04b20c7db2af6fd…
Attempting to fetch d171b92e3676d67b111459f80e877743bef7392f…
Attempting to fetch d171b92e3676d67b111459f80e877743bef7392f…
Attempting to fetch 7d227de7cc67ebe221f75ab7a09be444c050079c…
Attempting to fetch 77defe733251d546832c495c8f98382068f0a561…
Attempting to fetch 77defe733251d546832c495c8f98382068f0a561…
Attempting to fetch 0ab48ba0729a90524169a9cf816d535e27a508a4…
Attempting to fetch 0ab48ba0729a90524169a9cf816d535e27a508a4…
Attempting to fetch 1e872adef9491ca965719bf569f524dad28ebf83…
Attempting to fetch 1e872adef9491ca965719bf569f524dad28ebf83…
Attempting to fetch 9006475ccbdefcf87a0cdf3d71b5e408d0d3c110…
Attempting to fetch 9006475ccbdefcf87a0cdf3d71b5e408d0d3c110…
Attempting to fetch eef772d79f5721d2529cdc3fca8bfa17794254f5…
Attempting to fetch 8bb2fadddd5f4779d3d33f09880cd2fb3d415be8…
… and so on…

[AlexDaniel]

• [ ] you have to run greppable.t twice to make it pass its tests (it pulls perl6-all-modules on the first run and times out)

[AlexDaniel]

Oops. https://blog.github.com/2018-02-18-deprecation-notice-removing-anonymous-gist-creation/

• [ ] Find another alternative to anonymous gists

[JJ]

Maybe create an user and post from it?

[AlexDaniel]

Maybe create an user and post from it?

Maybe! However, Github does not allow to create more than two accounts, and I'm already over the limit. Also, everyone will be able to post any garbage gists under that account, so potentially it may get banned. So-o-o… I think we should be looking for non-github alternatives, but contributions are welcome.

[AlexDaniel]

FWIW I did a lot of improvements lately to this, so it's much better than it was. But still probably not good enough.

[AlexDaniel]

See also: https://github.com/perl6/whateverable/issues/323