server
server copied to clipboard
[badgerdb] vlog growing unbounded - consider adding GC and exposing options
I noticed that BadgerDB storage started growing unbounded (it was many files and several gigabytes quite quickly). Looks like the Badgerhold/Badger Storage example doesn't provide a way for the user to utilize GC correctly per Badgers docs. https://github.com/dgraph-io/badger/blob/main/db.go#L1225
Fairly sure the option tuning is probably useful to expose as well somehow -- for example ValueLogFileSize has a pretty large default
I didn't plumb up a full fix, just copied the default and started experimenting, but this showed that something like option tuning and GC can resolve the infinite growth modified a copy of the default to include some signalling and a regular GC call.
I would recommend at least adding the GC as a default. I would also suggest lowering some of the default or exposing them.
go func() {
ticker := time.NewTicker(5 * time.Minute)
defer ticker.Stop()
for {
select {
case <- ticker.C:
_ = h.db.Badger().RunValueLogGC(0.7)
continue
case <- h.doneCh:
return
}
}
}()