Storage Incentives Contract address change & overlay calculation update

[metacertain]

Summary

With the deployment of the redistribution-capable storage-incentives contract the bee node needs to rebuild the batch history based on the events of the new contract, upon upgrading to the new version using this contract's address. To make this transition smooth, a decision was made to make the release containing this also iterating the handshake protocol version number, to avoid interference of mixed versions with core protocols.

Changing sha3-256 to sha-256 or keccak-256 in the overlay address calculation is required to make the overlay address on-chain verifiably associated with an ethereum address. To be able to find an overlay similar to the previously calculated overlay address of the node (and to preserve topology of the network), the previous "transaction" field (originally referring to an on-chain transaction blockhash) is repurposed as a freely selectable "nonce".

Motivation

Transition for new contract as a dependency to future storage incentives redistribution

Implementation

The following changes were identified to be implemented for the smart contract change:

• Contract address change (0xTBD)
• Handshake protocol version incrementation
• Logic to accomodate for resyncing the batchstore from the events of the new contract (after disposing of obsolete data) (this probably needs to happen for the first time running the new version, if previous version has ever ran, similar to a migration)

The following changes are to be done for the overlay change:

• Remove verification of transaction provided in hive protocol etc, only calculate overlay based on provided nonce, network id, and ethereum address, if overlay is also communicated, verify it matches with the calculated one.
• Change hashing function used in overlay address to keccak256
• When node starts, get previous overlay if there was any
• Remove logic for making a transaction when starting a new node
• Logic to accomodate for mining a new overlay (if there was an old overlay) by changing "nonce" until PO(new overlay, old overlay) >= 31

Drawbacks

If the new contract doesn't have batches at the time of the breaking release, it could lead to a radius 0 leading to nodes trying to establish full connectivity in the network. This is to be avoided by having some (large) batches in store by the release by means of migration

[agazso]

Reading the drawbacks section: this means that the contract have to be deployed before the release, so that a large batch (or several) can be bought with it and some amount of data can be already uploaded by the time the new Bee version is released? How much data are we talking about?

[metacertain]

The contract has to be deployed before the release. In general I think you understand correctly. Well, migrating some already large valid batch with already uploaded content is also possible. If we have storage depth in place by the release, the merely existing batches would not necessary help against full connectivity, we would need actual uploaded content indeed, either migrated or fresh.

[metacertain]

So to get minimum radius 5 for example, we would need a batch bought with size of 16GB * 2^5 (512 GB) With storage depth, the uploaded content requirement is similar.

[ldeffenb]

So, would this stamp help? It's currently just short of 24% utilized with uploaded content already in the swarm:

  "batchID": "0e8366a6fdac185b6f0327dc89af99e67d9d3b3f2af22432542dc5971065c1df",
  "utilization": 3926,
  "usable": true,
  "label": "20210916-OSM-Map",
  "depth": 30,
  "amount": "441429710",
  "bucketDepth": 16,
  "blockNumber": 18125458,
  "immutableFlag": false,
  "exists": true,
  "batchTTL": 517107827

I just need to know what would be necessary to migrate it and when.