Reserve store implementation

[aloknerurkar]

Reserve Store

Used to track the chunks that are part of the node's area of responsibility.

Functionality

type ReserveStore interface {
        // Chunks function will provide the ChunkStore interface over the contents
        // of the ReserveStore. This will be done using provided Store and ChunkStore in
        // the transaction
        Chunks(storage.Tx) (storage.ChunkStore, error)

        // UnreserveBatch is used by the BatchStore to unreserve the chunks related to a
        // particular batch. 
        UnreserveBatch(storage.Tx, batchId []byte) error

        // Used by pull sync
        storage.PullSubscriber
}

Requirements

• Reserve store would maintain the existing postage and pullsync indexes.

type pullIndex struct {
        PO uint8
        BinID uin32
        Addr swarm.Address
        BatchId []byte
}

type postageChunks struct {
        BatchId []byte
        PO uint8
        Addr swarm.Address
}

type postageRadius struct {
        BatchId []byte
        Radius uint8
}

type postageIndex struct {
        BatchId []byte
        BatchIndex uint64
        Timestamp uint64
        Addr swarm.Address
}

• It has fixed capacity which is constant at build time. Users should not be able to change this value or run a node with a different value.

• Put operation going above capacity should replace chunk with lowest value. Check if we can do this using the current indexes or we need to change them or add a new one.

• Delete operation needs to provide mechanism for Cache to check if it wants to Cache the chunk.

Things to consider

• Iteration on all chunks would be required for the ReserveStore for storage incentives. We need to make sure current indexes are always atmost the capacity that we have set. This way this iteration will be constant time.

• Metrics around unreserved chunks, evicted lower value chunks, postageIndex related deletions etc.

[ldeffenb]

Please please please make sure the reserve store doesn't mess with local pinning as described in #3037

[aloknerurkar]

Please please please make sure the reserve store doesn't mess with local pinning as described in #3037

I will update you with more details. This is definitely the way!

[istae]

• The batchstore should just contain the list of batches, and calculate the reserve radius, and calls UnreserveBatch when the batch expires.
• The new reserve will maintain the storage radius and the radius notifier (to kademlia).
• The new reserve will take on the responsibilities of the depthmonitor.
• When reserve is full, it will iterate on the list of batches from the batchstore, then at the end calls bs.SetRadius()