paimon icon indicating copy to clipboard operation
paimon copied to clipboard
verified publisher icon apache

[core] Support MULTI_WRITE mode for KeyValueFileStoreWrite when multiple jobs are writing to one table simultaneously…

Open xiangyuf opened this issue 9 months ago • 1 comments

Purpose

Linked issue: close #5214

Tests

MergeTreeCompactManagerTest#testSyncOrphanFiles() KeyValueFileStoreWriteTest#testMultiWriteModeEnabled() KeyValueFileStoreWriteTest#testMultiWriteModeDisabled()

API and Format

Documentation

xiangyuf avatar Mar 17 '25 14:03 xiangyuf

@LinMingQiang I've discussed with @JingsongLi about this feature. We need to avoid continuously file scan in writer. So this pr will only be a PoC temporarily.

xiangyuf avatar Apr 02 '25 08:04 xiangyuf