flink icon indicating copy to clipboard operation
flink copied to clipboard

Published 20 hours ago • verified publisher icon apache

[FLINK-31238] [WIP] Use IngestDB to speed up Rocksdb rescaling recovery (rebased)

Open StefanRRichter opened this issue 1 year ago • 2 comments

Rebased and slightly refactored version of #23169.

StefanRRichter avatar Jan 05 '24 14:01 StefanRRichter

CI report:

  • 6d68d32d70fa445cb8e042d7ab14ca46c9496271 Azure: FAILURE
  • 05a7a0d9948c4f4e2502b69db9b4cdfcd2070aab Azure: PENDING
Bot commands The @flinkbot bot supports the following commands:
  • @flinkbot run azure re-run the last Azure build

flinkbot avatar Jan 05 '24 14:01 flinkbot

@rkhachatryan Can you take one more look? I have fixed all problems that I encountered in the original PR and also addressed your comments. Using TestStreamEnvironment.randomizeConfiguration seems a bit problematic because the RocksDB config keys are not visible in that module. We also still need to change to frocksDB version and the notice file once we have the release.

StefanRRichter avatar Jan 10 '24 15:01 StefanRRichter

One more thing, we should probably update the UI and show the new flag under Checkpoints / Configuration. But that can be a separate PR.

rkhachatryan avatar Jan 25 '24 17:01 rkhachatryan