substrate icon indicating copy to clipboard operation
substrate copied to clipboard

Asset Pallet: Support repeated destroys to safely destroy large assets

Open tonyalaribe opened this issue 2 years ago • 27 comments

Problem:

This is an attempt at solving #12275. At the moment when an asset with a very large number of accounts (sufficients) eg u32:Max is to be destroyed, the weight would simply be impossible to fit into a block.

Approach:

This PR will make it possible to call the destroy action repeatedly, and at each call account up to a certain max value would be destroyed, until all the accounts have been destroyed. The current approach involves breaking down the destroy extrinsic into four new extrinsics:

  • start_destroy : Mark the asset as being in a destroying state.
  • destroy_accounts : Can be called repeatedly until all accounts are destroyed. Only destroys max T:RemoveKeysLimit::get() keys at a time
  • destroy_approvals : Can be called repeatedly until all approvals are destroyed. Only destroys max T:RemoveKeysLimit::get() keys at a time
  • finish_destroy : Release the currency and finally delete the asset.

cumulus companion: https://github.com/paritytech/cumulus/pull/1742

tonyalaribe avatar Sep 20 '22 10:09 tonyalaribe

bot rebase

tonyalaribe avatar Oct 06 '22 13:10 tonyalaribe

Rebased

/cmd queue -c try-runtime $

tonyalaribe avatar Oct 13 '22 13:10 tonyalaribe

@tonyalaribe Could not find start of command (" $ ")

command-bot[bot] avatar Oct 13 '22 13:10 command-bot[bot]

/cmd queue -c try-runtime

tonyalaribe avatar Oct 13 '22 13:10 tonyalaribe

@tonyalaribe Could not find start of command (" $ ")

command-bot[bot] avatar Oct 13 '22 13:10 command-bot[bot]

/cmd queue -c try-runtime $ on-runtime-upgrade

tonyalaribe avatar Oct 13 '22 13:10 tonyalaribe

@tonyalaribe https://gitlab.parity.io/parity/mirrors/substrate/-/jobs/1948310 was started for your command "$PIPELINE_SCRIPTS_DIR/try-runtime-bot.sh" on-runtime-upgrade. Check out https://gitlab.parity.io/parity/mirrors/substrate/-/pipelines?page=1&scope=all&username=group_605_bot to know what else is being executed currently.

Comment /cmd cancel 8-513b477e-3e58-4b6f-94af-f335a1ee3af8 to cancel this command or /cmd cancel to cancel all commands in this pull request.

command-bot[bot] avatar Oct 13 '22 13:10 command-bot[bot]

@tonyalaribe Command "$PIPELINE_SCRIPTS_DIR/try-runtime-bot.sh" on-runtime-upgrade has finished. Result: https://gitlab.parity.io/parity/mirrors/substrate/-/jobs/1948310 has finished. If any artifacts were generated, you can download them from https://gitlab.parity.io/parity/mirrors/substrate/-/jobs/1948310/artifacts/download.

command-bot[bot] avatar Oct 13 '22 13:10 command-bot[bot]

/cmd queue -c try-runtime $ on-runtime-upgrade live wss://westmint-rpc.polkadot.io

tonyalaribe avatar Oct 13 '22 15:10 tonyalaribe

@tonyalaribe https://gitlab.parity.io/parity/mirrors/substrate/-/jobs/1949306 was started for your command "$PIPELINE_SCRIPTS_DIR/try-runtime-bot.sh" on-runtime-upgrade live wss://westmint-rpc.polkadot.io. Check out https://gitlab.parity.io/parity/mirrors/substrate/-/pipelines?page=1&scope=all&username=group_605_bot to know what else is being executed currently.

Comment /cmd cancel 10-d455d19c-e866-4006-87f5-17e0e3db9292 to cancel this command or /cmd cancel to cancel all commands in this pull request.

command-bot[bot] avatar Oct 13 '22 15:10 command-bot[bot]

@tonyalaribe Command "$PIPELINE_SCRIPTS_DIR/try-runtime-bot.sh" on-runtime-upgrade live wss://westmint-rpc.polkadot.io has finished. Result: https://gitlab.parity.io/parity/mirrors/substrate/-/jobs/1949306 has finished. If any artifacts were generated, you can download them from https://gitlab.parity.io/parity/mirrors/substrate/-/jobs/1949306/artifacts/download.

command-bot[bot] avatar Oct 13 '22 16:10 command-bot[bot]

/cmd queue -c try-runtime $ on-runtime-upgrade live --uri wss://westmint-rpc.polkadot.io

tonyalaribe avatar Oct 13 '22 16:10 tonyalaribe

@tonyalaribe https://gitlab.parity.io/parity/mirrors/substrate/-/jobs/1949309 was started for your command "$PIPELINE_SCRIPTS_DIR/try-runtime-bot.sh" on-runtime-upgrade live --uri wss://westmint-rpc.polkadot.io. Check out https://gitlab.parity.io/parity/mirrors/substrate/-/pipelines?page=1&scope=all&username=group_605_bot to know what else is being executed currently.

Comment /cmd cancel 11-3bbac858-977e-45ad-bebf-5caca01e87f0 to cancel this command or /cmd cancel to cancel all commands in this pull request.

command-bot[bot] avatar Oct 13 '22 16:10 command-bot[bot]

@tonyalaribe Command "$PIPELINE_SCRIPTS_DIR/try-runtime-bot.sh" on-runtime-upgrade live --uri wss://westmint-rpc.polkadot.io has finished. Result: https://gitlab.parity.io/parity/mirrors/substrate/-/jobs/1949309 has finished. If any artifacts were generated, you can download them from https://gitlab.parity.io/parity/mirrors/substrate/-/jobs/1949309/artifacts/download.

command-bot[bot] avatar Oct 13 '22 16:10 command-bot[bot]

/cmd queue -c try-runtime $ on-runtime-upgrade live --uri wss://statemine-rpc.polkadot.io

tonyalaribe avatar Oct 13 '22 16:10 tonyalaribe

@tonyalaribe https://gitlab.parity.io/parity/mirrors/substrate/-/jobs/1949370 was started for your command "$PIPELINE_SCRIPTS_DIR/try-runtime-bot.sh" on-runtime-upgrade live --uri wss://statemine-rpc.polkadot.io. Check out https://gitlab.parity.io/parity/mirrors/substrate/-/pipelines?page=1&scope=all&username=group_605_bot to know what else is being executed currently.

Comment /cmd cancel 12-0cd3065b-723c-45f1-9831-243aa8862ed1 to cancel this command or /cmd cancel to cancel all commands in this pull request.

command-bot[bot] avatar Oct 13 '22 16:10 command-bot[bot]

@tonyalaribe Command "$PIPELINE_SCRIPTS_DIR/try-runtime-bot.sh" on-runtime-upgrade live --uri wss://statemine-rpc.polkadot.io has finished. Result: https://gitlab.parity.io/parity/mirrors/substrate/-/jobs/1949370 has finished. If any artifacts were generated, you can download them from https://gitlab.parity.io/parity/mirrors/substrate/-/jobs/1949370/artifacts/download.

command-bot[bot] avatar Oct 13 '22 16:10 command-bot[bot]

/cmd queue -c try-runtime $ on-runtime-upgrade live --uri wss://statemine-rpc.polkadot.io:9944

tonyalaribe avatar Oct 13 '22 16:10 tonyalaribe

@tonyalaribe https://gitlab.parity.io/parity/mirrors/substrate/-/jobs/1949414 was started for your command "$PIPELINE_SCRIPTS_DIR/try-runtime-bot.sh" on-runtime-upgrade live --uri wss://statemine-rpc.polkadot.io:9944. Check out https://gitlab.parity.io/parity/mirrors/substrate/-/pipelines?page=1&scope=all&username=group_605_bot to know what else is being executed currently.

Comment /cmd cancel 13-bcce39eb-5e26-4bc7-b2e5-c80a51403bec to cancel this command or /cmd cancel to cancel all commands in this pull request.

command-bot[bot] avatar Oct 13 '22 16:10 command-bot[bot]

@tonyalaribe looks good to me. can you solve the failing checks

muharem avatar Oct 13 '22 16:10 muharem

@tonyalaribe Command "$PIPELINE_SCRIPTS_DIR/try-runtime-bot.sh" on-runtime-upgrade live --uri wss://statemine-rpc.polkadot.io:9944 has finished. Result: https://gitlab.parity.io/parity/mirrors/substrate/-/jobs/1949414 has finished. If any artifacts were generated, you can download them from https://gitlab.parity.io/parity/mirrors/substrate/-/jobs/1949414/artifacts/download.

command-bot[bot] avatar Oct 13 '22 16:10 command-bot[bot]

You can only call the try-runtime bot in Polkadot (Cumulus as well maybe, not sure).

ggwpez avatar Oct 14 '22 10:10 ggwpez

You can only call the try-runtime bot in Polkadot (Cumulus as well maybe, not sure).

@ggwpez Thanks. I didn't know that.

tonyalaribe avatar Oct 14 '22 11:10 tonyalaribe

/cmd queue -c bench-bot $ pallet dev pallet_assets

tonyalaribe avatar Oct 14 '22 11:10 tonyalaribe

@tonyalaribe https://gitlab.parity.io/parity/mirrors/substrate/-/jobs/1951994 was started for your command "$PIPELINE_SCRIPTS_DIR/bench-bot.sh" pallet dev pallet_assets. Check out https://gitlab.parity.io/parity/mirrors/substrate/-/pipelines?page=1&scope=all&username=group_605_bot to know what else is being executed currently.

Comment /cmd cancel 18-b96e5dec-42d5-47d6-82bb-b151fc25fb45 to cancel this command or /cmd cancel to cancel all commands in this pull request.

command-bot[bot] avatar Oct 14 '22 11:10 command-bot[bot]

@tonyalaribe Command "$PIPELINE_SCRIPTS_DIR/bench-bot.sh" pallet dev pallet_assets has finished. Result: https://gitlab.parity.io/parity/mirrors/substrate/-/jobs/1951994 has finished. If any artifacts were generated, you can download them from https://gitlab.parity.io/parity/mirrors/substrate/-/jobs/1951994/artifacts/download.

command-bot[bot] avatar Oct 14 '22 11:10 command-bot[bot]

bot merge

tonyalaribe avatar Nov 15 '22 09:11 tonyalaribe

We are currently integrating this change into Moonbeam and do not understand the need for this migration: https://github.com/paritytech/substrate/pull/12310/files#diff-b4f24881ddef27f3416692a99eef0ffdd5af704ec20ff3c3033124f4a818cc63

The new AssetStatus enum is scale encoded the same way as a bool for the first 2 variants, so what is the purpose of this migration? Can we consider not including it in our runtime upgrades?

librelois avatar Feb 10 '23 17:02 librelois

Hi @librelois, to the best of my knowlege, i would play safe and run the migration if you already store asset metadata via this pallet. The migration basically sets a default AssetStatus for the existing assets. I'm not very sure you would be able to decode them otherwise.

tonyalaribe avatar Feb 13 '23 10:02 tonyalaribe