linera-protocol icon indicating copy to clipboard operation
linera-protocol copied to clipboard

Fast (history-based?) hashing for KeyValueStoreView

Open MathieuDutSik opened this issue 2 years ago • 1 comments

Currently, hashing a collection requires hashing all the data sequentially. This is problematic for KeyValueStoreView that could be large for some applications.

One simple approach is to update the hash continuously by hashing the chain of write (insert, delete, etc) operations themselves.

Caveat: after this PR, two KV stores could contain the same data but hash differently if their history of writes is different.

Question: Should we remove the inefficient hashing in MapView, LogView, etc?

MathieuDutSik avatar Oct 02 '23 18:10 MathieuDutSik

Usually, these kind of problems require a merkle trie. It can be used for maps for O(log n) hashing, while for LogView a simple vector of historical hashes is sufficient and is also O(1). But this approach is only justifiable at large scale if values stored are significantly larges than 32 bytes of the hashes

usagi32 avatar Feb 16 '25 06:02 usagi32