redpanda icon indicating copy to clipboard operation
redpanda copied to clipboard
verified publisher icon redpanda-data

chunk-starvation

Open emaxerrno opened this issue 4 years ago • 0 comments

Depending on a particular workload there might be a situation in which the chunk appender, might take on full 32MB and wait to synchronize all 32MB in 16KB chunks.

strategy to improve

  1. for the quota reservation of the memory, we should use 128KB
  2. 16KB for follow up chunks
  3. eager fallocation without synchronization (waiting for all chunks to finish

cc: @rkruze @mmaslankaprv @Lazin

emaxerrno avatar Jan 31 '21 02:01 emaxerrno