facebook
User-defined timestamp support for `DeleteRange()`
Summary: Add user-defined timestamp support for range deletion. The new API is DeleteRange(opt, cf, begin_key, end_key, ts). Most of the change is to update the comparator to compare without timestamp. Other than that, major changes are
- internal range tombstone data structures (
FragmentedRangeTombstoneList,RangeTombstone, etc.) to store timestamps. - Garbage collection of range tombstones and range tombstone covered keys during compaction.
- Get()/MultiGet() to return the timestamp of a range tombstone when needed.
- Get/Iterator with range tombstones bounded by readoptions.timestamp.
- timestamp crash test now issues DeleteRange by default.
Test plan:
- Added unit test:
make check - Stress test:
python3 tools/db_crashtest.py --enable_ts whitebox --readpercent=57 --prefixpercent=4 --writepercent=25 -delpercent=5 --iterpercent=5 --delrangepercent=4 - Ran
db_benchto measure regression when timestamp is not enabled. The tests are for write (with some range deletion) and iterate with DB fitting in memory:./db_bench--benchmarks=fillrandom,seekrandom --writes_per_range_tombstone=200 --max_write_buffer_number=100 --min_write_buffer_number_to_merge=100 --writes=500000 --reads=500000 --seek_nexts=10 --disable_auto_compactions -disable_wal=true --max_num_range_tombstones=1000. Did not see consistent regression in no timestamp case.
| micros/op | fillrandom | seekrandom |
|---|---|---|
| main | 2.58 | 10.96 |
| PR 10661 | 2.68 | 10.63 |
@cbi42 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.
@cbi42 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.
@cbi42 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.
@cbi42 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.
@cbi42 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.
@cbi42 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.
@cbi42 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.
@cbi42 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.