kvrocks icon indicating copy to clipboard operation
kvrocks copied to clipboard

Bump rocksdb to v9.1.1

Open aleksraiden opened this issue 1 year ago • 9 comments
trafficstars

Bump rocksdb to 9.1.0. Full release notes - https://github.com/facebook/rocksdb/releases/tag/v9.1.0

Key features:

  • On file systems that support storage level data checksum and reconstruction, retry SST block reads for point lookups, scans, and flush and compaction if there's a checksum mismatch on the initial read
  • Some enhancements and fixes to experimental Temperature handling features
  • WriteBatchWithIndex now supports wide-column point lookups via the GetEntityFromBatch API
  • Best-effort recovery (best_efforts_recovery == true) may now be used together with atomic flush (atomic_flush == true). The all-or-nothing recovery guarantee for atomically flushed data will be upheld.
  • Remove deprecated option bottommost_temperature, already replaced by last_level_temperature
  • Added new PerfContext counters for block cache bytes read - block_cache_index_read_byte, block_cache_filter_read_byte, block_cache_compression_dict_read_byte, and block_cache_read_byte.
  • Deprecate experimental Remote Compaction APIs
  • Defined a new PerfLevel kEnableWait to measure time spent by user threads blocked in RocksDB other than mutex, such as a write thread waiting to be added to a write group, a write thread delayed or stalled etc.
  • RateLimiter's API no longer requires the burst size to be the refill size
  • Fixed kBlockCacheTier reads to return Status::Incomplete
  • Fixed a data race in WalManager
  • Re-enable the recycle_log_file_num option in DBOptions for kPointInTimeRecovery WAL recovery mode, which was previously disabled due to a bug in the recovery logic. This option is incompatible with WriteOptions::disableWAL. A Status::InvalidArgument() will be returned if disableWAL is specified.
  • Java API multiGet() variants now take advantage of the underlying batched multiGet() performance improvements.

Bump v9.1.1 bugfix:

  • Fixed a regression when ColumnFamilyOptions::max_successive_merges > 0 where the CPU overhead for deciding whether to merge could have increased unless the user had set the option ColumnFamilyOptions::strict_max_successive_merges

aleksraiden avatar Apr 18 '24 19:04 aleksraiden

Update: bump rocksdb to 9.1.1 (bugfix release - https://github.com/facebook/rocksdb/releases/tag/v9.1.1)

  • Fixed a regression when ColumnFamilyOptions::max_successive_merges > 0 where the CPU overhead for deciding whether to merge could have increased unless the user had set the option ColumnFamilyOptions::strict_max_successive_merges

aleksraiden avatar Apr 23 '24 08:04 aleksraiden

@git-hulk @PragmaTwice I need your superpower here when you have time. Please take a look at the build error. I tried to figure out the reason but failed. Just a regular RocksDB code was changed between 9.1.0 and 9.1.1 but the error somehow appeared.

torwig avatar Apr 30 '24 04:04 torwig

@torwig Sure, thanks for your investigation. I'm not sure if it's related to the musl library since other CI cases are good. Will take a look when I get time, maybe @PragmaTwice is an expert on this 😎

git-hulk avatar Apr 30 '24 04:04 git-hulk

Maybe can help us - https://github.com/jemalloc/jemalloc/issues/778 and https://github.com/google/tcmalloc/issues/150 (but tmalloc)

aleksraiden avatar May 02 '24 08:05 aleksraiden

I'am fixed this bug, create at separate PR

aleksraiden avatar May 02 '24 17:05 aleksraiden

Now we can try update from upstream and check it

aleksraiden avatar May 03 '24 07:05 aleksraiden

Hmm, not working, like a mystic )

aleksraiden avatar May 03 '24 17:05 aleksraiden

https://github.com/apache/kvrocks/pull/2298#issuecomment-2105726276

I've meet this issue in rocksdb, sigh...Maybe we should shift to a new version later?

mapleFU avatar May 11 '24 12:05 mapleFU

#2298 (comment)

I've meet this issue in rocksdb, sigh...Maybe we should shift to a new version later?

Yeah, I'am waiting an next release of rocksdb at a week... and update PR

aleksraiden avatar May 12 '24 10:05 aleksraiden

succeeded by #2327.

PragmaTwice avatar May 31 '24 14:05 PragmaTwice