pulsar-client-go
pulsar-client-go copied to clipboard
aynsc send [hasSpace()] control has bad performance when send large message
Expected behavior
- when i send large message body,the client has a queue to hold it and send by batch, there are two triggers: 1. message
count lager then
maxMessages, OR 2. messge body sum is larger then 'BatchingMaxSize' - Know the codes are 【bc.numMessages+1 <= bc.maxMessages && bc.buffer.ReadableBytes()+msgSize <= uint32(bc.maxBatchSize)】,and it will brings somemany Flush opertion when i send large message body
Actual behavior
- 【unable to add message to batch】will called when send message body size nearly 'BatchingMaxSize'
Steps to reproduce
- send body size lager then 3M
- make send fast than 1000 qps
System configuration
Pulsar version: 2.9.0
https://github.com/apache/pulsar-client-go/pull/792 might fix this problem. It guarantees at least one message to be added to a batch that will be sent.
[unable to add message to batch] is hard to produce otherwise. Because there is an internal flush already before the second Add() to batch, I believe such error only occurs when the first flush fails. Do you see any error messages prior?