bee icon indicating copy to clipboard operation
bee copied to clipboard

increase the reservesize to 2^23 chunks (~33 GB)

Open istae opened this issue 1 year ago • 4 comments

the mainnet has recently seen an increase in storage radius to radius 10. this has doubled the neighborhood count to 1024 and left MANY neighborhoods running on fewer than 2 nodes, in some cases, zero peers, leading to data loss. An immediate effect of increasing the reserve size is the decreasing of the radius so that neighborhood count is halved and neighborhoods are populated sufficiently again.

Also, it should be the long term goal of the team to balance bee resource and hardware requirements, and the number of nodes required to store data in the network, and in this regard, reservesizes are currently too small, and needs to be increased to improve this balance.

Tasks:

  • [ ] double reserve size
  • [ ] tweak the redistribution phase durations so that sufficient time is given for the sampler to finish the new reserve size of chunks OR come up with a new mechanism to make sample hash duration independent of the reserver size

istae avatar Sep 25 '23 12:09 istae

proposal https://hackmd.io/bqoWRODNQReNHYoBe3oAsw

istae avatar Sep 25 '23 17:09 istae

proposal https://hackmd.io/bqoWRODNQReNHYoBe3oAsw

please make these documents public accessible. Following these links I receive a 403

tmm360 avatar Sep 26 '23 13:09 tmm360

We expect this will improve the batch utilisation issue. Before proceeding with implementation of this, we need green lights from the research team. @ct-ctrl @zelig

nikipapadatou avatar Oct 16 '23 09:10 nikipapadatou

@nikipapadatou This will improve the issue of small neighborhood populations.

istae avatar Oct 16 '23 23:10 istae