rocksdb
rocksdb copied to clipboard
facebook
db_bench support user defined timestamp
Note: Please use Issues only for bug reports. For questions, discussions, feature requests, etc. post to dev group: https://groups.google.com/forum/#!forum/rocksdb or https://www.facebook.com/groups/rocksdb.dev
Expected behavior
db_bench support enabling user defined timestamp
Actual behavior
some errors encountered in while benchmarking this feature with the tool
Steps to reproduce the behavior
Note: Please use Issues only for bug reports. For questions, discussions, feature requests, etc. post to dev group: https://groups.google.com/forum/#!forum/rocksdb or https://www.facebook.com/groups/rocksdb.dev ./db_bench --num=10000000 --disable_wal --value_size=16 --report_interval_seconds=1 --db=/tmp/timestamp --statistics --benchmarks=fillrandom --delayed_write_rate=$((12810241024)) --max_background_flushes=4 --max_background_compactions=8 --threads=16 -user_timestamp_size=8 --compression_type=none
./db_bench --num=10000000 --disable_wal --value_size=16 --report_interval_seconds=1 --db=/tmp/timestamp --statistics --benchmarks=readrandom --delayed_write_rate=$((12810241024)) --cache_size=$(( 4 *1024 * 1024)) --max_write_buffer_number=6 --max_background_flushes=4 --max_background_compactions=8 -user_timestamp_size=8 --threads=1 --compression_type=none --use_existing_db --duration=60 --read_with_latest_user_timestamp
On the write path: the keys that are created are without a timestamp. The command batch.clear() also reset the value of default_cf_ts_sz_ and the db_bench does not set it back On the read path: only a small fraction of the keys, this is suspected to be an issue with the tool instead of the feature's own bug
@jowlyzhang : thanks for assigning this to yourself. Do you know when you could pick this issue up?
@jowlyzhang : thanks for assigning this to yourself. Do you know when you could pick this issue up?
I don't think I will be able to check this soon, are you interested in taking it?
We are blocked on this issue -- but don't think we have the capacity to take this ourselves as we aren't as familiar with this codebase/path
We are blocked on this issue -- but don't think we have the capacity to take this ourselves as we aren't as familiar with this codebase/path
Thank you for this context. I will check this out soon by end of next week.
LDB scan and get do not work as well. The issue in LDB is that the caller must give timestamp for new iterator & Get (unlike snapshot where no snapshot means the latest). LDB can not create a timestamp without a knowledge on the cooperator. I think that in order to make ldb works the simplest way is to support null timestamp (get the latest version in get ; get all the versions in scan) in the read options.
@jowlyzhang -- wanted to check in again and see if this is planned to be addressed
I was actually recently working on this, the ./ldb get type of commands, like multiget, scan, get_entity etc should have read timestamp support after this PR #12641
I'm currently working on dump, wal_dump and idump for similar support and I will later work on the db_bench one.
Amazing ! Thank you so much we will stay tuned and test it out after
On Wed, May 22 2024 at 1:48 PM, Yu Zhang < @.*** > wrote:
I was actually recently working on this, the./ldb get type of commands, like multiget , scan , get_entity etc should have read timestamp support after this PR #12641 ( https://github.com/facebook/rocksdb/pull/12641 )
I'm currently working on dump , wal_dump and idump for similar support and I will later work on the db_bench one.
— Reply to this email directly, view it on GitHub ( https://github.com/facebook/rocksdb/issues/12233#issuecomment-2125417161 ) , or unsubscribe ( https://github.com/notifications/unsubscribe-auth/AALD6HTSQ453VDKSG3IHIBLZDTLAPAVCNFSM6AAAAABBYYUYZCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMRVGQYTOMJWGE ). You are receiving this because you are subscribed to this thread. Message ID: <facebook/rocksdb/issues/12233/2125417161 @ github. com>