immudb icon indicating copy to clipboard operation
immudb copied to clipboard

Published 20 hours ago • verified publisher icon codenotary

Optimize memory footprint of immudb when using large number of entries per transaction

Open tomekkolo opened this issue 3 years ago • 0 comments

Running immudb with a large number of entries per transaction results in large demand for RAM.

How to reproduce

  • Start immudb with 20 480 entries per transaction setting.

  • Immudb requires 16GB of RAM to be run

  • Start immudb with 1024 entries per transaction setting

  • Immudb requires around 500 MB of RAM.

Acceptance criteria:

  • immudb does not preallocate memory but allocates on demand

References:

Buffer: https://github.com/codenotary/immudb/blob/caddadad5e48a5cb7475e317a01d1aef91468ca9/embedded/store/immustore.go#L319

Where used: https://github.com/codenotary/immudb/blob/caddadad5e48a5cb7475e317a01d1aef91468ca9/embedded/store/immustore.go#L1041

It is copied to dynamically allocated buffer anyway: https://github.com/codenotary/immudb/blob/caddadad5e48a5cb7475e317a01d1aef91468ca9/embedded/store/immustore.go#L1118

tomekkolo avatar Feb 15 '22 14:02 tomekkolo