zenfs
zenfs copied to clipboard
westerndigitalcorporation
Garbage collection
Add an option to enable garbage collection in zenfs.
The garbage collection, if enabled, kicks in under 20% capacity left and cleans out zones with 1% valid data in them to start off with, increasing to 40% at 99% capacity.
This scheme reduces space amplification down to as little as 5% (from ~10-15%) in my testing.
I've addressed all comments(except the refactoring things I believe) and increased the gc aggressiveness a bit (slope 2 -> 3). It looks stable and performing nicely, (sysbench and db_bench tests give promising results), but it might need more fine tuning, so let's keep this on a branch until we've finished evaluating this,
Hi guys! A few days ago I tested this garbage collection draft and it stuck. I guess the GC threshold is not good. Considering a case where all zones have valid and invalid data at the same time, ZenFS maybe can not find a empty zone to collect the garbage. Will you take a look at this? Please tell me if my assumption is wrong! Thanks very much!
Hi guys! A few days ago I tested this garbage collection draft and it stuck. I guess the GC threshold is not good. Considering a case where all zones have valid and invalid data at the same time, ZenFS maybe can not find a empty zone to collect the garbage. Will you take a look at this? Please tell me if my assumption is wrong! Thanks very much!
Hi Haltz,
Thanks for testing it out. Could you give more information on what happened with the hang(how to reproduce this? ) ? What were you testing on it with (Hardware) ? What application were you running ? Did you look at the logs at /tmp/znefs_*.log ?
I will give more infomation(maybe a few days later). I was testing this branch on FEMU(a emulator based on QEMU). RocksDB was in the guest virtual machine and using a emulated ZNS SSD as backend storage device.
I have some follow-up patches to this pull request(including migration bug fixes), but will merge this and then submit the rest as separate PRs.