silkworm
silkworm copied to clipboard
Db format for Account/Storage History indexes
Current implementation (compatible with erigon stable) defines the indexes schema as :
k : key + bitmap_shard_upper_limit (BE uint64_t)
v : bitmap data
where key
is :
- for
AccountHistory
the accountaddress
- for
StorageHistory
the composite ofcontract address + location
Need to evaluate if is worth to change the table definitions to dupsorted
(aka multivalue) as :
k : key
v : bitmap_shard_upper_limit (BE uint64_t) + bitmap data
There should be minimal changes in the reads of historical accounts/storages and lot less key duplications
As @AskAlexSharov points out in multi-value mode the size of v
is constrained to keySizeLimit
(I honestly didn't think about that)
Need to stat how many shards per account/location and average shard data size to find a sweet spot.