paimon icon indicating copy to clipboard operation
paimon copied to clipboard

[compaction] Filter out invalid splits to improve flink database compaction efficiency

Open zhourui999 opened this issue 1 year ago • 0 comments

Purpose

Linked issue: close #xxx Filter out invalid splits to improve flink database compaction efficiency

(1) Referring to the pickFullComparison method of CompactStrategy, filter out whether dataFiles is empty or all files in dataFiles are at the max-level image

Tests

API and Format

Documentation

zhourui999 avatar Apr 18 '24 07:04 zhourui999