tokio
tokio copied to clipboard
tokio-rs
[docs] Codec Encoder and Decoder traits do not describe buffer management
Version All
Platform n/a
Description When implementing a codec, it's common to see the BytesMut struct grow to hundreds of MB that can't be reclaimed. This is a combination of issues, but the main is that there seems to be no way for a https://docs.rs/bytes/latest/bytes/struct.BytesMut.html#method.truncate to release memory from it's allocation.
The docs of BytesMut describe all the places where capacity is grown or preserved, but there is no mention of when capacity can be freed.
Nor does tokio-util describe how to ensure that the BytesMut doesn't continue to grow to excessive sizes during operation.
For example https://docs.rs/tokio-util/latest/tokio_util/codec/trait.Decoder.html#tymethod.decode says:
"An optimal buffer management strategy minimizes reallocations and over-allocations."
But gives no information on how to achieve this, nor does BytesMut clearly describe this.
As a result, this leaves developers confused how to manage memory (at best) and at worst, causes memory leaks of hundreds of MB (at worst).
Can this documentation about BytesMut management be improved?
An example of this is when you have a Codec where most messages are small, but one can be a large request, that single large request can blow your buffer size up and then can't be free even if you call .clear() .advance() or .split(). So there seems to be no way to manage a buffer like this without some kind of unbounded growth, so we need advice on how to do this properly.
Okay, having now read the source code to BytesMut there is from my understanding no way to free capacity. This will cause a problem with long lived connections/workloads where the BytesMut will grow to the size of the largest request and can not be released after that, unless the connection is terminated and reset.
There needs to be a way to free space in a BytesMut, and it must be documented clearly as a method to reduce the capacity of the buffer.
The buffer should never be larger than twice the largest message you have received. If you have rare large messages, then the codec's implementation can replace the BytesMut with a shrunk one.
The important point here is that there is no obvious way to actually clear this (such as the swap solution you proposed) nor is there any warning to the codec users that this kind of buffer size expansion can occur. There needs to be a significantly improved set of documentation about how memory is managed in these buffers, not just in how they are allocating, but also how to reclaim space in the conditions where this kind of memory consumption can grow and never be released.
Can the docs for codec be improved to actually demonstrate things like how the memory grows, how to clear/truncate/advance, and also to show how to free memory with the swap technique clearly explaining that there is no other way to free the memory?