polkadot icon indicating copy to clipboard operation
polkadot copied to clipboard

Pending Rococo benchmarks

Open brunopgalvao opened this issue 3 years ago • 11 comments

Follow up to https://github.com/paritytech/polkadot/pull/5052

Generate Rococo weights for the following:

The following weights were copied from Polkadot and need to be re-generated for Rococo

Additionally, XCM needs to be added as a benchmark for Rococo.

  • [ ] Add XCM benchmark and generate weights

brunopgalvao avatar Mar 22 '22 18:03 brunopgalvao

Please also have a look at the comments from the first MR, since there seems to be some mixup.

ggwpez avatar Mar 22 '22 21:03 ggwpez

Adding XCM benchmarks is not trivial, and probably something that should be put off for later.

These last few benchmarks we can handle.

shawntabrizi avatar Mar 22 '22 21:03 shawntabrizi

I'm going to add rococo to the existing weights benchmarking pipelines anyway which will take care of regenerating the weights

coderobe avatar Apr 02 '22 18:04 coderobe

Checking in here, is this good to close now cc @brunopgalvao @al3mart ?

nuke-web3 avatar Jun 12 '22 21:06 nuke-web3

This can be closed when this issue is closed: https://github.com/paritytech/polkadot/pull/5617 as it has the XCM benchmarks in there.

brunopgalvao avatar Jun 13 '22 13:06 brunopgalvao

This can be closed when this issue is closed: #5617 as it has the XCM benchmarks in there.

We don't want pallet_session benchmarks to be a blocker of #5617, runtime upgrade can happen with the current weights.

XCM weights will be added.

NachoPal avatar Jun 24 '22 07:06 NachoPal

My findings about the state of Rococo benchmarks after #5617 is merged.

  • Issue with pallet_parahacins_paras_inherent https://github.com/paritytech/polkadot/pull/4305

    2022-07-11 19:22:21 panicked at 'assertion failed: `(left == right)`
      left: `1`,
     right: `101`', /Users/nacho/Desktop/PARITY/Repos/polkadot/runtime/parachains/src/paras_inherent/benchmarking.rs:119:9
    

    Last commit in the file is 8 months ago, so it seems it has been failing since then

  • These two have to be run manually and not with the script (10 months old in Kusama) https://github.com/paritytech/polkadot/pull/3511

    • pallet_collective_technical_commitee
    • pallet_collective instead of pallet_collective_council
  • pallet_session was removed because it will fail → https://substrate.stackexchange.com/questions/3273/benchmark-session-pallet-fails-without-pallet-staking

  • pallet_recovey generated weights but not added (not in Kusama either)

NachoPal avatar Jul 26 '22 16:07 NachoPal

@ggwpez anything we can do here?

brunopgalvao avatar Aug 01 '22 14:08 brunopgalvao

Yea I think I know whats going on. Fix should be trivial, will open an MR.

ggwpez avatar Aug 01 '22 16:08 ggwpez

Ah actually not sure anymore. I thought it was easy, but not that much anymore.
Should this not already be fixed if the Rococo runtime is the same as Kusama?
Somehow the benches still fail on the udpated-rococo-runtime MR...

ggwpez avatar Aug 02 '22 11:08 ggwpez

Ah actually not sure anymore. I thought it was easy, but not that much anymore. Should this not already be fixed if the Rococo runtime is the same as Kusama? Somehow the benches still fail on the udpated-rococo-runtime MR...

We don't want to block the PR because of the benchmarks. That PR has been waiting already too long. In addition, since those benchmarks apparently never worked, it doesn't seem critical to have them fixed before the runtime upgrade.

We'd like to address them in a different PR.

NachoPal avatar Aug 02 '22 11:08 NachoPal