SMT
SMT copied to clipboard
Growing size of DB with same keys update
Hi, It seems the library does not removed the nodes, which is already added to the db, during the updating. The old nodes are only removed in the live cache, rather than in the db. So if one node is commit to the db, it will stay there forever. For example, if I run the code:
keys := getFreshData(10000, 32)
for i := 0; i < 30; i++ {
values := getFreshData(10000, 32)
smt.Update(keys, values)
smt.Commit()
}
The size of the db keeps growing, even the keys do not change.
I tried to fix the issue by adding:
var rootCopy [32]byte
copy(rootCopy[:], root)
switch {
case len(lkeys) == 0 && len(rkeys) > 0:
// s.db.removedNode stores all the root, which is modified in this update, as a map
if rootCopy != [32]byte{0} && height%4 == 0 {
s.db.removeMux.Lock()
s.db.removedNode[rootCopy] = []byte{0}
s.db.removeMux.Unlock()
}
s.updateRight(lnode, rnode, root, keys, values, batch, iBatch, height, ch)
case len(lkeys) > 0 && len(rkeys) == 0:
if rootCopy != [32]byte{0} && height%4 == 0 {
s.db.removeMux.Lock()
s.db.removedNode[rootCopy] = []byte{0}
s.db.removeMux.Unlock()
}
s.updateLeft(lnode, rnode, root, keys, values, batch, iBatch, height, ch)
default:
if rootCopy != [32]byte{0} && height%4 == 0 {
s.db.removeMux.Lock()
s.db.removedNode[rootCopy] = []byte{0}
s.db.removeMux.Unlock()
}
s.updateParallel(lnode, rnode, root, lkeys, rkeys, lvalues, rvalues, batch, iBatch, height, ch)
}
in the
func (s *Trie) update(root []byte, keys, values, batch [][]byte, iBatch, height int, ch chan<- (mresult))
When Commit(), the key in the s.db.removedNode will be removed. And in my test scenario, the size of db no longer grows. Do you think it's a valid fix?