titan
titan copied to clipboard
Rewrite valid keys by ingesting SST files during GC
Since the write callback is removed by #121 , it's possible to generate SST files of valid keys and to ingest these SST files by IngestExternalFile
The benchmark of https://github.com/tikv/titan/pull/121 shows Ingest may hurt read performance, see
dbbench (./titandb_bench --key_size="16" --value_size="256" --cache_size="4294967296" --titan_min_blob_size="0" --num="10000000" --numdistinct="10000" --benchmarks="fillseq,overwrite,readrandom" --threads="6" --titan_max_background_gc="6"):
gc_rewrite_mode = kDefault
fillseq: 99.3 MB/s; overwrite: 23.2 MB/s; readrandom: 130.5 MB/s
gc_rewrite_mode = kMerge
fillseq: 102.5 MB/s; overwrite: 59.3 MB/s; readrandom: 122.8 MB/s
gc_rewrite_mode = kFastIngest
fillseq: 101.1 MB/s; overwrite: 61.8 MB/s; readrandom: 82.4 MB/s
go-ycsb (max-background-gc=4, threads=256, readcount=1kw, operationcount=2kw)
gc_rewrite_mode = kDefault
update ops 5w -> 4.53w, read ops 12w
gc_rewrite_mode = kMerge
update ops 5w -> 4.58w, read ops 11w
gc_rewrite_mode = kFastIngest
update ops 5w -> 4.7w, read ops 7w