Add badger index cache
Introduce a badgerDB (https://github.com/dgraph-io/badger) as a disk index cache. It could be a middle layer cache between in-memory and remote cache (in-memory -> (badger) -> Memcached or Redis).
There are two loop in the badger index cache, retentionLoop and diskStatUpdateLoop.
retentionLoop: periodically run badgerDB GC to sink disk space.
diskStatUpdateLoop: exports metric badger_value_log_space_available_bytes and badger_value_log_space_available_bytes track amount of disk space left on the value and key log used in Badger at mount point in bytes.
Which issue(s) this PR fixes: Fixes #6241
Checklist
- [X] Tests updated
- [ ] Documentation added
- [x]
CHANGELOG.mdupdated - the order of entries should be[CHANGE],[FEATURE],[ENHANCEMENT],[BUGFIX]
@yeya24
Using Size, we can track bytes usage of vlog, lsm. But, I should do more digging to delete keys when some threshold reached.
This is a very interesting PR.. at some point i was thinking on doing something similar.
@SungJin1212 were you able to do some load test on this to see how it performs compared with the other caching solutions?
@alanprot Thank you for your interest in my work. I haven't done the load test yet, but I was thinking that the load tests are necessary. Which performance aspects are essential in the load test do you think?
This issue has been automatically marked as stale because it has not had any activity in the past 60 days. It will be closed in 15 days if no further activity occurs. Thank you for your contributions.