ion-java icon indicating copy to clipboard operation
ion-java copied to clipboard

Provide API for PooledBlockAllocator management

Open honourosis opened this issue 1 year ago • 1 comments

The current PooledBlockAllocator implementation relies on a single PooledBlockAllocatorProvider instance. The current PooledBlockAllocator implementation has an upper limit on the number of free blocks that can be reused, but there is no way to free unused blocks.

Let's take for instance this scenario:

  1. We need to serialize a very large file, for example more than 100 Mb
  2. Using PooledBlockAllocator with blockSize = 32768 results in the creation of 2048 blocks
  3. Once the serialization is complete, we don't need 2048 blocks because all the following serializations will only use a few dozen blocks
  4. As a result, we have a lot of pre-allocated blocks that are never used

It would be nice to have some API to explicitly control the PooledBlockAllocator. Also, this problem can be solved by using some heuristics based on "allocation pressure". For example, if we have 2048 blocks, but have used no more than 100 blocks in the last 1 minute, we can be sure that reducing half of the free blocks would be safe.

honourosis avatar Mar 13 '24 11:03 honourosis

Hello, thanks for submitting this request. We will look into this as soon as possible.

linlin-s avatar Mar 18 '24 20:03 linlin-s