ssv icon indicating copy to clipboard operation
ssv copied to clipboard

Get decided by range optimization

Open amirylm opened this issue 3 years ago • 0 comments

Is your feature request related to a problem? Please describe.

Currently, when some node is asked for decided message in a specific range, it reads all the desired messages one by one: https://github.com/bloxapp/ssv/blob/a4fa01c462b013b43e0e34511d7b7788bd1219c1/storage/kv/badger.go#L66

This results a memory leak when it get enough hits, what usually happens in high loads, e.g. history sync.

Describe the solution you'd like

The items should be fetched in batches with a single transaction instead of a new transaction for each item.

A possible way to do it is to add a method to badger interface:

GetMany(prefix []byte, keys ...[]byte) (basedb.Obj, map[string]error)

The prefix should include the identifier of some specific validator and role, and keys is a list all the desired sequence numbers. it makes it possible to read all the values with a single badger transaction.

Describe alternatives you've considered

Additional context

amirylm avatar Nov 30 '21 12:11 amirylm