substrate
substrate copied to clipboard
Force base weights to be the minimum only when the intercept is negative
Followup of https://github.com/paritytech/substrate/pull/11806 and https://github.com/paritytech/substrate/pull/12386
This PR changes two things:
-
The base weights are now forced to be equal to the minimum execution time only if the intercept of the linear regression is negative. Looking at https://github.com/paritytech/substrate/pull/12325 there are still issues with doing this unconditionally and not all of our benchmarks are ready for it. So this PR is a compromise that forces the base weights to be the minimum only when it's necessary (when the base weight would otherwise be negative), and when it's not it just uses whatever intercept was calculated. This should hopefully cut down on the number of benchmarks we need to fix.
-
The minimum execution time is now printed out in the comments generated along with the weights. We often use benchmarks to see whether something got slower/faster, however the weights generated through linear regression are not really an appropriate way to do that. With the base weights always being the minimum execution times we could have used those to see how slow/fast something actually is (since they were not a product of the linear regression), but since after this PR those base weights won't always be equal to the minimum execution times anymore we just move this information into the comments. This doesn't really change anything functionally, and it's just there as extra information for whoever is reading the file with the weights.
/cmd queue -c bench-bot $ pallet dev pallet_contracts
@ggwpez https://gitlab.parity.io/parity/mirrors/substrate/-/jobs/1948761 was started for your command "$PIPELINE_SCRIPTS_DIR/bench-bot.sh" pallet dev pallet_contracts. Check out https://gitlab.parity.io/parity/mirrors/substrate/-/pipelines?page=1&scope=all&username=group_605_bot to know what else is being executed currently.
Comment /cmd cancel 9-3e34654e-a477-4628-8238-8e7897430dfd to cancel this command or /cmd cancel to cancel all commands in this pull request.
@ggwpez Command "$PIPELINE_SCRIPTS_DIR/bench-bot.sh" pallet dev pallet_contracts has finished. Result: https://gitlab.parity.io/parity/mirrors/substrate/-/jobs/1948761 has finished. If any artifacts were generated, you can download them from https://gitlab.parity.io/parity/mirrors/substrate/-/jobs/1948761/artifacts/download.
Update: fixed in b27f42c804fe9d755856ca9ec5cf61c24924dada
The benchbot failed with Error: Io(Custom { kind: Other, error: "Failed to access JSON data." }).
I hope it was not this one https://github.com/paritytech/pipeline-scripts/pull/82
PS: This error seems to come from Handlebar.
/cmd queue -c bench-bot $ pallet dev pallet_assets
@ggwpez https://gitlab.parity.io/parity/mirrors/substrate/-/jobs/1972669 was started for your command "$PIPELINE_SCRIPTS_DIR/bench-bot.sh" pallet dev pallet_assets. Check out https://gitlab.parity.io/parity/mirrors/substrate/-/pipelines?page=1&scope=all&username=group_605_bot to know what else is being executed currently.
Comment /cmd cancel 28-43972c72-8f82-4e7f-9da0-5d87650901dd to cancel this command or /cmd cancel to cancel all commands in this pull request.
@ggwpez Command "$PIPELINE_SCRIPTS_DIR/bench-bot.sh" pallet dev pallet_assets has finished. Result: https://gitlab.parity.io/parity/mirrors/substrate/-/jobs/1972669 has finished. If any artifacts were generated, you can download them from https://gitlab.parity.io/parity/mirrors/substrate/-/jobs/1972669/artifacts/download.
/cmd queue -c bench-bot $ pallet dev pallet_uniques
@ggwpez https://gitlab.parity.io/parity/mirrors/substrate/-/jobs/1973082 was started for your command "$PIPELINE_SCRIPTS_DIR/bench-bot.sh" pallet dev pallet_uniques. Check out https://gitlab.parity.io/parity/mirrors/substrate/-/pipelines?page=1&scope=all&username=group_605_bot to know what else is being executed currently.
Comment /cmd cancel 29-0fef2a97-b085-4e14-90af-ada602a12367 to cancel this command or /cmd cancel to cancel all commands in this pull request.
@ggwpez Command "$PIPELINE_SCRIPTS_DIR/bench-bot.sh" pallet dev pallet_uniques has finished. Result: https://gitlab.parity.io/parity/mirrors/substrate/-/jobs/1973082 has finished. If any artifacts were generated, you can download them from https://gitlab.parity.io/parity/mirrors/substrate/-/jobs/1973082/artifacts/download.
bot merge