iotdb
iotdb copied to clipboard
[IOTDB-3164] Manage the memory of cross space compaction in write memory controller
See IOTDB-3164.
This PR is one part of the compaction memory management. It does the following jobs:
- Allocate memory before cross space compaction, and release the memory after the execution.
- Remove the
RewriteCompactionFileSelector
, which is deduplicated withRewriteCrossSpaceCompactionSelector
.
To complete the compaction memory management, we need to complete the following jobs:
- Manage the memory usage of unsequence inner space compaction, which uses read point performer.
- Design and implement the memory estimator for read chunk performer.
- Manage the memory usage of sequence inner space comapction, which uses read chunk performer.
Coverage increased (+0.02%) to 42.338% when pulling ad73693498b616abc847d1e8988464175c373ee7 on IOTDB-3164-O into 37d6cfed32b4aa2858c898dcb012b824f3ab041e on master.
I use IoTDB benchmark tests changed after the performance, and the master (commit: a8f757d6e4fecc2251d2042a025960bf29c6e0f3) are compared. The result of the comparison is shown in the figure below. In terms of average performance, the write performance after the change decreases by about 2.8%.
master branch
with memory management for cross space compaction
https://apache-iotdb.feishu.cn/docx/doxcnuh5hy6ee9Qb7uLCYXkFDdc