prometheus-kafka-adapter icon indicating copy to clipboard operation
prometheus-kafka-adapter copied to clipboard

Published 20 hours ago • verified publisher icon Telefonica

error: Local: Queue full, msg: couldn't produce massage in kafka

Open golangav opened this issue 3 years ago • 4 comments

Hello

This error is also an error report when the network is not available, which is easy to mislead. Restart and return to normal

Can you distinguish it

https://github.com/confluentinc/confluent-kafka-go/issues/346

image image

golangav avatar Mar 26 '22 02:03 golangav

Hi @golangav, thanks for opening this issue. What do you recommend us to do when the queue is full?

palmerabollo avatar Mar 27 '22 15:03 palmerabollo

Hello, can you add a queue

or batch.size linger.ms

golangav avatar Mar 29 '22 22:03 golangav

@palmerabollo Hello, can you add a queue https://github.com/confluentinc/confluent-kafka-dotnet/issues/703 image

queue. buffering. Max.kbytes to specify how much information can be cached. The default is 400MB, and the maximum can be increased to 2097151kb, that is, about 2GB (because the cache is not 100% of your messages, but also has key or other meta information, so the actual available can not reach 2GB).

queue. buffering. Max.messages to specify how many messages can be cached. The default is 100000. That's enough.

Errors such as buffererror: Local: queue full can be avoided by changing these two properties and optimizing the process control of the program.

golangav avatar Mar 29 '22 22:03 golangav