fluency icon indicating copy to clipboard operation
fluency copied to clipboard

Introduce strict buffer retention size property

Open sokomishalov opened this issue 2 years ago • 3 comments

It would be useful in the case of using fluent-bit with low default buffering settings.

sokomishalov avatar Sep 02 '21 11:09 sokomishalov

What do you mean by "strict buffer retention size"?

komamitsu avatar Sep 04 '21 07:09 komamitsu

Thanks for the quick reply! Well, I've found your reply here that buffer chunk retention size is not actually strict so it has to be set a bit less than it actually could be and does not even save us from exceeding fluentd/fluent-bit buffering limit. So I suppose it could be useful if it is implementable at all.

sokomishalov avatar Sep 04 '21 17:09 sokomishalov

I see. I got the point.

Yes. It sounds a good improvement. I would try it later.

komamitsu avatar Sep 05 '21 01:09 komamitsu

Sorry, I may have read your comment wrongly. Setting BufferChunkRetentionSize to less than the buffer limit of Fluentd / Fluent Bit should work. I'll update Fluency README to mention the relationship between Fluentd / Fluent Bit buffer size and Fluency's BufferChunkRetentionSize.

komamitsu avatar Sep 10 '22 07:09 komamitsu

Ah, I remember that indeed there is room to improve to make it more strict. Let me try.

komamitsu avatar Sep 10 '22 08:09 komamitsu