Upload store implementation

[aloknerurkar]

Upload Store

Used for deferred uploads. This component would manage the pushIndex.

Functionality

type UploadStore interface {
        // Upload will provide a Putter which will add the pushIndex
        // with the tag provided and store the chunk in the ChunkStore
        // The entire Putter operation will be guarded by the transaction
        Upload(tx storage.Tx, tag uint64) (storage.Putter, error)
        // PushSubscriber is the existing functionality used by Pusher to get chunks
        // to push. Here we iterate on the pushIndex.
        storage.PushSubscriber
        // Used by Pusher to read the chunks with TagId and Delete the Chunk from
        // UploadStore
        Chunks(tx storage.Tx) (storage.ChunkStore, error)
}

Requirements

• Maintain pushIndex to iterate on for PushSubscriber. Define the pushIndex for Store.

// key can be created using the TimeStamp + Addr. The timestamp will provide an order to iterate.
type pushIndex struct {
        Timestamp   uint64
        Addr        swarm.Address
        TagId       uint64
}

• During Delete of Chunk, we need to cleanup the pushIndex. Upload store would also need a mechanism to provide to Cache to hook into deletion and react to it.

• During interaction with Pusher, we will need to decorate the TagId on the chunk while reading. This will be used by tags service to mark the chunk as synced.

Things to consider

• Metrics related to UploadStore. For eg. no. of chunks uploaded, synced etc.