docs: produce message size rfc

[fraidev]

TL;DR

batch_size producer config must not reject large records, just send them directly.

Create a new max_request_size producer config that must reject large messages. I am using max_request_size because Kafka uses max.request.size but we can change it to other config name.

Compression sizes should not be used for these producer configs.

Related:

• https://github.com/infinyon/fluvio/issues/4161
• https://github.com/infinyon/fluvio/pull/4195

[sehz]

Never mind previous comments. Here is clarifying semantics.

max_request_size applies to pre-compression record size so can be applied to per record constrain. batch-size applies to post-compression limit (or underlying raw records limit). We can set max-request-size high for default case and bump up batch-size as well.

[fraidev]

max_request_size applies to pre-compression record size so can be applied to per record constrain. batch-size applies to post-compression limit (or underlying raw records limit).

I think that we should use the compressed size for batch-size as we only compress the whole batch. Not each record of the batch.

We can set max-request-size high for default case and bump up batch-size as well.

Sure, Kafka for example uses 16kb for batch_size like us and 1mb for max.request.size