How to handle EOF for rare messages

[un000]

Versions

Please specify real version numbers or git SHAs, not just "Latest" since that changes fairly regularly.

Sarama	Kafka	Go
v1.27.2	2.5.0	1.16.4

Configuration

What configuration values are you using for Sarama and Kafka?

	// common config
	cc := sarama.NewConfig()
	cc.ClientID, _ = os.Hostname()
	cc.ClientID = p.config.Group + "_" + cc.ClientID + "_" + strconv.Itoa(id)
	cc.ChannelBufferSize = 1 << 15
	cc.Version = kafkaVersion
	cc.Producer.Retry.Max = 0
	cc.Producer.Return.Errors = true
	cc.Producer.Return.Successes = false
	cc.Producer.RequiredAcks = sarama.WaitForAll
	cc.Producer.MaxMessageBytes = 4 * 1024 * 1024
	cc.Producer.Compression = sarama.CompressionSnappy
	cc.Producer.Timeout = 15 * time.Second
	cc.Net.MaxOpenRequests = 64
	cc.Net.DialTimeout = 15 * time.Second
	cc.Net.KeepAlive = time.Minute

Logs

"error":"EOF"
or
"error":"write tcp 10.233.67.54:53092->10.233.67.100:9092: write: broken pipe"

Problem Description

Hi, there is a trouble. We are trying to produce a message 5 times per minute. EOF errors happen frequently. I think that heartbeat is not working for some reasons. What should we configure? Ty.

[reaVen-]

Try setting KeepAlive to a lower value, something in-between might close the connection faster

[un000]

https://github.com/Shopify/sarama/issues/760

maybe related to this

[github-actions[bot]]

Thank you for taking the time to raise this issue. However, it has not had any activity on it in the past 90 days and will be closed in 30 days if no updates occur. Please check if the main branch has already resolved the issue since it was raised. If you believe the issue is still valid and you would like input from the maintainers then please comment to ask for it to be reviewed.