lua-resty-kafka icon indicating copy to clipboard operation
lua-resty-kafka copied to clipboard

Published 20 hours ago • verified publisher icon doujiang24

openresty-lua and kafka server BufferOverflow error

Open dev-omniwyse opened this issue 6 years ago • 2 comments

we are running openresty-lua and Kafka servers in different data centres, openresty handles 20K requests per second. whenever openresty-lua sending data in async model to Kafka server, It,s throwing BufferOverflow error.

How to overcome this error?

dev-omniwyse avatar May 03 '19 12:05 dev-omniwyse

@dev-omniwyse the network between the openresty and kafka may be a bit slow, you can increase the max_buffering and batch_num.

you can also try to use the concurrency branch, it can send data faster, but it haven't been tested fully. https://github.com/doujiang24/lua-resty-kafka/tree/concurrency

doujiang24 avatar May 03 '19 12:05 doujiang24

Hi doujiang24,

Thanks for the reply! Where exactly we need to do changes to increase the max_bufferig and batch_num.

dev-omniwyse avatar May 07 '19 10:05 dev-omniwyse