joystream icon indicating copy to clipboard operation
joystream copied to clipboard

No unbound collections in stored objects

Open Lezek123 opened this issue 2 years ago • 1 comments

  1. Make sure we don't store any collections that have an unbound number of items/entries (either directly or as part of a struct / other stored type)
  2. Make sure we always use worst-case-scenario sized objects for all extrinsics that load/modify those objects (in the runtime state) in the benchmarks

The problem with storing unbound collections is that the size of the object stored by the runtime (for example Channel, Video, CuratorGroup etc.) has a significant impact on all operations performed on this object.

For example: the cost of set_curator_group_status for an empty curator group (no curators and no perrmisions_by_level) is ~4 times lower (not counting the cost of read and write operations) than the cost of updating a wrost-case-scenario sized curator group (curators.len() == T::MaxNumberOfCuratorsPerGroup, permissions_by_level.len() == T::MaxKeysPerCuratorGroupPermissionsByLevelMap)

This means that when benchmarking extrinsics that load/modify given object in the runtime state we should always use an object with a worst-case-scenario possible size for a given type. When an object contains an unbound collection this is either not possible or very hard to estimate.

Some examples of unbound collections we're currently storing:

  • Channel.collaborators
  • Channel.data_objects
  • Video.data_objects

Some examples of benchmarking where we don't use worst-case-scenario size when the extrinsic is operating on an object that contains a collection:

  • Storage::accept_pending_data_objects - the bag is being loaded from the runtime state, but the bag.distributed_by collection is not pre-populated with the worst-case-scenario number of items in the benchmarks
  • ...

Those are just a few examples I've been able to catch by quickly glancing at the code, there is probably a lot more of those.

┆Issue is synchronized with this Asana task by Unito

Lezek123 avatar Jul 15 '22 14:07 Lezek123

https://github.com/Joystream/joystream/issues/4046#issuecomment-1199179759

Lezek123 avatar Aug 01 '22 07:08 Lezek123