rocksdb icon indicating copy to clipboard operation
rocksdb copied to clipboard

Fix a corruption bug in `CreateColumnFamilyWithImport()`

Open cbi42 opened this issue 10 months ago • 3 comments

Summary: when importing files from multiple CFs into a new CF, we were reusing the epoch numbers assigned by the original CFs. This means L0 files in the new CF can have the same epoch number (assigned originally by different CFs). While CreateColumnFamilyWithImport() requires each original CF to have disjoint key range, after an intra-l0 compaction, we still can end up with L0 files with the same epoch number but overlapping key range. This PR attempt to fix this by reassigning epoch numbers when importing multiple CFs.

Test plan: a new repro unit test. Before this PR, it fails with

[ RUN      ] ImportColumnFamilyTest.AssignEpochNumberToMultipleCF
db/import_column_family_test.cc:1048: Failure
db_->WaitForCompact(o)
Corruption: force_consistency_checks(DEBUG): VersionBuilder: L0 files of same epoch number but overlapping range #44 , smallest key: '6B6579303030303030' seq:511, type:1 , largest key: '6B6579303031303239' seq:510, type:1 , epoch number: 3 vs. file #36 , smallest key: '6B6579303030313030' seq:401, type:1 , largest key: '6B6579303030313939' seq:500, type:1 , epoch number: 3

cbi42 avatar May 01 '24 20:05 cbi42

@cbi42 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

facebook-github-bot avatar May 01 '24 22:05 facebook-github-bot

@cbi42 has updated the pull request. You must reimport the pull request before landing.

facebook-github-bot avatar May 04 '24 00:05 facebook-github-bot

@cbi42 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

facebook-github-bot avatar May 04 '24 00:05 facebook-github-bot

@cbi42 Thanks very much for the quick fix. This PR looks good to me

mayuehappy avatar May 06 '24 06:05 mayuehappy

@cbi42 merged this pull request in facebook/rocksdb@6fdc4c52823d0b32bb18321b0d4b14ab70d09e92.

facebook-github-bot avatar May 06 '24 18:05 facebook-github-bot