nspcc-dev
Allow configurable LevelDB options
Is your feature request related to a problem? Please describe.
The current configuration for LevelDB creates too many files. It's possible to decrease the number of files by changing the CompactionTableSize on LevelDB. The large amount of files has a negative impact on our IT infrastructure.
Describe the solution you'd like
Level DB has some parameters that can be tweaked, such as WriteBufferSize, BlockSize, BlockCacheCapacity, CompactionTableSize, CompactionL0Trigger, and OpenFilesCacheCapacity. These values should be configurable through the config file.
Describe alternatives you've considered
I've tried running 'db dump', but it takes a long time to complete.
Additional context
I've implemented this feature here, without any kind of test. https://github.com/linkd-academy/neo-go/commit/8ca4a311c01f7627967bf6fe818083b02271bb96
Result with 5GB CompactionTableSize (may not be ideal, 1 GB could be better):
Changing some parameters made it run faster on my macOS. My computer was barely usable during synchronization.
Don't forget to add labels!
-
enhancement
I've implemented this feature here, without any kind of test. https://github.com/linkd-academy/neo-go/commit/8ca4a311c01f7627967bf6fe818083b02271bb96
Patches are always welcomed! Open a PR, and we'll review&merge eventually.
Btw, boltdb also could also have some love.
There are InitialMmapSize and AllocSize which regulate how big the database is initially and how fast it grows.
It may be important, because to grow a database we need to remap it, and we cannot remap if there is an existing session iterator, so all transactions hang. I have reproduced it multiple times in privnet scenario with heavy iterator usage.
With these 2 settings the probability of such event can be made much smaller.
neofs-node also used to configure MaxBatchSize and MaxBatchDelay but I have found them not that useful for neo-go.
