Jesse Szwedko

> Perhaps the first thing to do would be aware in components, to make this more of a manage-able process. Is it possible for each component to understand the memory...

Strong agree. I think we can support AVRO and Protobuf with runtime definitions first and add support for schema registry later. https://github.com/apache/avro/tree/master/lang/rust seems like it supports runtime parsing of AVRO.

Agreed, thanks @Daniel599! Closed by https://github.com/vectordotdev/vector/pull/18019

Suggestions from a user here: https://github.com/vectordotdev/vector/issues/20284

> I think also docs for buffering and batching should be extended as it is also very difficult to understand relation between buffer and batch limits. Agreed, the docs could...

> @jszwedko thank you, that is what I barely remember from Discord discussion some time ago. Still I think it might be beneficial for buffer to be larger than batch...

Also mentioned in discord again here: https://discord.com/channels/742820443487993987/746070604192415834/879421822922260492

> Any thoughts about being able to set memory limit through config file? This would be nice. It is just a difficult feature to implement as all memory allocations now...

Agreed, thanks for the idea @micoq ! I think we can support it both as a codec on sources as well as a VRL function. cc/ @pablosichert who has been...

@ryn9 just curious if you see this same behavior with the new disk buffer implementation (should be the default in 0.26.0).