logstash icon indicating copy to clipboard operation
logstash copied to clipboard

PQ can exceed queue.max_bytes

Open kaisecheng opened this issue 2 years ago • 1 comments

Currently, the queue size is the sum of the event size per tail page and head page. Some pages may have 100kb event, some may have 30mb. However, pages are immutable and consume a fixed size, default 64mb, on disk. So, the size calculation should take the capacity not the event size. Hence, the current PQ can consume more than the queue.max_bytes

kaisecheng avatar Sep 28 '22 10:09 kaisecheng

Check comment for the queue max size assumption

kaisecheng avatar Mar 13 '24 12:03 kaisecheng