kafka
kafka copied to clipboard
Producer is crashed when 100000 records are pushed
I have a simple producer and trying to add 100000 json records continuously (streaming data) to a topic but after some point of time, the producer is crashed. How to push huge volume of stream of JSON records to a single topic using producer?
Do you have exact exception message and trace when it crashed?
This is one of the errors without any change in my code. I will post the gc full error message with my code change.
2017-01-23T16:20:02.989Z ERROR no-kafka-client Metadata request failed: NoKafkaConnectionError [127.0.0.1:9092]: Error: read ECONNRESET { [NoKafkaConnectionError: Error: read ECONNRESET] name: 'NoKafkaConnectionError', server: '127.0.0.1:9092', message: 'Error: read ECONNRESET' }
This looks like your Kafka server has closed connection. Probably because you exceeded message size or something else. Please check your Kafka server logs.