rocksdb
rocksdb copied to clipboard
Fix a corruption bug in `CreateColumnFamilyWithImport()`
Summary: when importing files from multiple CFs into a new CF, we were reusing the epoch numbers assigned by the original CFs. This means L0 files in the new CF can have the same epoch number (assigned originally by different CFs). While CreateColumnFamilyWithImport() requires each original CF to have disjoint key range, after an intra-l0 compaction, we still can end up with L0 files with the same epoch number but overlapping key range. This PR attempt to fix this by reassigning epoch numbers when importing multiple CFs.
Test plan: a new repro unit test. Before this PR, it fails with
[ RUN ] ImportColumnFamilyTest.AssignEpochNumberToMultipleCF
db/import_column_family_test.cc:1048: Failure
db_->WaitForCompact(o)
Corruption: force_consistency_checks(DEBUG): VersionBuilder: L0 files of same epoch number but overlapping range #44 , smallest key: '6B6579303030303030' seq:511, type:1 , largest key: '6B6579303031303239' seq:510, type:1 , epoch number: 3 vs. file #36 , smallest key: '6B6579303030313030' seq:401, type:1 , largest key: '6B6579303030313939' seq:500, type:1 , epoch number: 3
@cbi42 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.
@cbi42 has updated the pull request. You must reimport the pull request before landing.
@cbi42 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.
@cbi42 Thanks very much for the quick fix. This PR looks good to me
@cbi42 merged this pull request in facebook/rocksdb@6fdc4c52823d0b32bb18321b0d4b14ab70d09e92.