starrocks
starrocks copied to clipboard
Lz4compress may alloc exceed memory
Enhancement
size_t max_compressed_len(size_t len) const override {
return std::max(LZ4F_compressBound(len, &_s_preferences), LZ4F_compressFrameBound(len, &_s_preferences));
}
for lz4 sream compress, no need to alloc more memory of LZ4F_compressBound(len, &_s_preferences) size.
it may be slow when alloc memory larger then 16KB in tcmalloc, and may cause unnecessary shrink to fit size.
Whether it is possible to allocate appropriate memory according to the actual usage
We have marked this issue as stale because it has been inactive for 6 months. If this issue is still relevant, removing the stale label or adding a comment will keep it active. Otherwise, we'll close it in 10 days to keep the issue queue tidy. Thank you for your contribution to StarRocks!