polkadot-sdk

polkadot-sdk copied to clipboard

take over #2941 Weights compare

bot help

Here's a link to docs

bot bench substrate-pallet --pallet=pallet_contracts

"$PIPELINE_SCRIPTS_DIR/commands/bench/bench.sh" --subcommand=pallet --runtime=dev --target_dir=substrate --pallet=pallet_contracts was queued.

Comment bot cancel 12-b6340cf3-cf20-426e-8418-9ef2f2662dcc to cancel this command or bot cancel to cancel all commands in this pull request.

@pgherveou Command "$PIPELINE_SCRIPTS_DIR/commands/bench/bench.sh" --subcommand=pallet --runtime=dev --target_dir=substrate --pallet=pallet_contracts has finished. Result:

ValidationError: "id" is required
ValidationError: "id" is required
{"message":{"base":["Reference not found"]}}

bot bench substrate-pallet --pallet=pallet_contracts

"$PIPELINE_SCRIPTS_DIR/commands/bench/bench.sh" --subcommand=pallet --runtime=dev --target_dir=substrate --pallet=pallet_contracts was queued.

Comment bot cancel 13-1cdb3131-e249-4e7f-8d3d-fce894ed640e to cancel this command or bot cancel to cancel all commands in this pull request.

@pgherveou Command "$PIPELINE_SCRIPTS_DIR/commands/bench/bench.sh" --subcommand=pallet --runtime=dev --target_dir=substrate --pallet=pallet_contracts has finished. Result:

ValidationError: "id" is required
ValidationError: "id" is required
{"message":{"base":["Reference not found"]}}

bot bench substrate-pallet --pallet=pallet_contracts

@mordamax https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/5523429 was started for your command "$PIPELINE_SCRIPTS_DIR/commands/bench/bench.sh" --subcommand=pallet --runtime=dev --target_dir=substrate --pallet=pallet_contracts. Check out https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/pipelines?page=1&scope=all&username=group_605_bot to know what else is being executed currently.

Comment bot cancel 16-f37b5da1-9691-4c81-a0fb-ce26fe0701c9 to cancel this command or bot cancel to cancel all commands in this pull request.

@mordamax Command "$PIPELINE_SCRIPTS_DIR/commands/bench/bench.sh" --subcommand=pallet --runtime=dev --target_dir=substrate --pallet=pallet_contracts has finished. Result: https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/5523429 has finished. If any artifacts were generated, you can download them from https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/5523429/artifacts/download.

Hey @pgherveou , I saw that the benchmarks have finished but I'm not sure how to interpret the results. Are there any findings regarding possible execution improvements here?

@Robbepop @athei not sure why we commented on the previous PR https://github.com/paritytech/polkadot-sdk/pull/2941#pullrequestreview-1836267022

We need to change the benchmarks to compile the code in its setup so that when executing the contract it is only execution.

Why would we want to do that when we benchmark the extrinsic that is suppose to measure the whole thing

bot bench substrate-pallet --pallet=pallet_contracts

@pgherveou https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/5610549 was started for your command "$PIPELINE_SCRIPTS_DIR/commands/bench/bench.sh" --subcommand=pallet --runtime=dev --target_dir=substrate --pallet=pallet_contracts. Check out https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/pipelines?page=1&scope=all&username=group_605_bot to know what else is being executed currently.

Comment bot cancel 26-9ce4acbb-15c8-4149-bb27-c5c0da14c727 to cancel this command or bot cancel to cancel all commands in this pull request.

@pgherveou Command "$PIPELINE_SCRIPTS_DIR/commands/bench/bench.sh" --subcommand=pallet --runtime=dev --target_dir=substrate --pallet=pallet_contracts has finished. Result: https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/5610549 has finished. If any artifacts were generated, you can download them from https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/5610549/artifacts/download.

bot bench substrate-pallet --pallet=pallet_contracts

@pgherveou "$PIPELINE_SCRIPTS_DIR/commands/bench/bench.sh" --subcommand=pallet --runtime=dev --target_dir=substrate --pallet=pallet_contracts (https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/5613353) was cancelled in https://github.com/paritytech/polkadot-sdk/pull/3679#issuecomment-2014642626

bot cancel 1-220b369f-437d-46dd-b2ab-c07221359aaa

bot bench substrate-pallet --pallet=pallet_contracts

@pgherveou Command "$PIPELINE_SCRIPTS_DIR/commands/bench/bench.sh" --subcommand=pallet --runtime=dev --target_dir=substrate --pallet=pallet_contracts has finished. Result: https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/5613353 has finished. If any artifacts were generated, you can download them from https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/5613353/artifacts/download.

@pgherveou https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/5614252 was started for your command "$PIPELINE_SCRIPTS_DIR/commands/bench/bench.sh" --subcommand=pallet --runtime=dev --target_dir=substrate --pallet=pallet_contracts. Check out https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/pipelines?page=1&scope=all&username=group_605_bot to know what else is being executed currently.

Comment bot cancel 2-8052407e-12ec-4b6f-b166-c3cf1f1ccb68 to cancel this command or bot cancel to cancel all commands in this pull request.

@pgherveou Command "$PIPELINE_SCRIPTS_DIR/commands/bench/bench.sh" --subcommand=pallet --runtime=dev --target_dir=substrate --pallet=pallet_contracts has finished. Result: https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/5614252 has finished. If any artifacts were generated, you can download them from https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/5614252/artifacts/download.

Not sure if I'm reading the new gas benchmarks correct but it seems that quite a few methods have regressed >10%.

bot bench substrate-pallet --pallet=pallet_contracts

@pgherveou https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/5866476 was started for your command "$PIPELINE_SCRIPTS_DIR/commands/bench/bench.sh" --subcommand=pallet --runtime=dev --target_dir=substrate --pallet=pallet_contracts. Check out https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/pipelines?page=1&scope=all&username=group_605_bot to know what else is being executed currently.

Comment bot cancel 4-72d78cec-0877-4617-9dbd-a3cf611db4b3 to cancel this command or bot cancel to cancel all commands in this pull request.

@pgherveou Command "$PIPELINE_SCRIPTS_DIR/commands/bench/bench.sh" --subcommand=pallet --runtime=dev --target_dir=substrate --pallet=pallet_contracts has finished. Result: https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/5866476 has finished. If any artifacts were generated, you can download them from https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/5866476/artifacts/download.

bot bench substrate-pallet --pallet=pallet_contracts

@pgherveou https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/5868516 was started for your command "$PIPELINE_SCRIPTS_DIR/commands/bench/bench.sh" --subcommand=pallet --runtime=dev --target_dir=substrate --pallet=pallet_contracts. Check out https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/pipelines?page=1&scope=all&username=group_605_bot to know what else is being executed currently.

Comment bot cancel 5-f281cd4b-82bc-4bed-8491-2184246d219b to cancel this command or bot cancel to cancel all commands in this pull request.

@pgherveou Command "$PIPELINE_SCRIPTS_DIR/commands/bench/bench.sh" --subcommand=pallet --runtime=dev --target_dir=substrate --pallet=pallet_contracts has finished. Result: https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/5868516 has finished. If any artifacts were generated, you can download them from https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/5868516/artifacts/download.