automq-for-rocketmq icon indicating copy to clipboard operation
automq-for-rocketmq copied to clipboard

Published 20 hours ago • verified publisher icon AutoMQ

Reduce heap usage for small record

Open SCNieh opened this issue 1 year ago • 1 comments

the heap usage of BlockCache and LogCache will significantly increase due to the large number of StreamRecordBatch when record size is small

SCNieh avatar Dec 14 '23 04:12 SCNieh

  1. Reduce BlockCache heap usage by directly put data block into cache block and parsing it into StreamRecordBatch on demand
  2. Reduce LogCache heap usage by directly put encoded StreamRecordBatch into StreamCache and decode for indexing on binary search (or cache indices for acceleration by sacrificing more heap memory)

SCNieh avatar Feb 19 '24 09:02 SCNieh