rocksdb icon indicating copy to clipboard operation
rocksdb copied to clipboard

LOG more info on oldest snapshot and sequence numbers

Open pdillinger opened this issue 3 years ago • 1 comments

Summary: The info LOG file does not currently give any direct information about the existence of old, live snapshots, nor how to estimate wall time from a sequence number within the scope of LOG history. This change addresses both with:

  • Logging smallest and largest seqnos for generated SST files, which can help associate sequence numbers with write time (based on flushes).
  • Logging oldest_snapshot_seqno for each compaction, which (along with that seqno info) helps us to determine how much old data might be kept around for old (leaked?) snapshots. Including the date here I thought might be excessive.

I wanted to log the date and seqno of the oldest snapshot with periodic stats, but the current structure of the code doesn't really support that because DumpDBStats doesn't have access to the DB object.

Test Plan: manual inspect LOG from KEEP_DB=1 ./db_basic_test --gtest_filter=*CompactBetweenSnapshots*

pdillinger avatar Aug 01 '22 20:08 pdillinger

@pdillinger has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

facebook-github-bot avatar Aug 01 '22 20:08 facebook-github-bot

@pdillinger has updated the pull request. You must reimport the pull request before landing.

facebook-github-bot avatar Aug 12 '22 18:08 facebook-github-bot

@pdillinger has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

facebook-github-bot avatar Aug 12 '22 18:08 facebook-github-bot