segmentio
Before writing Client request all Topics from the Cluster
I am not considering this a bug, rather something to understand why as I have faced bad symptoms. Describe the bug For every WriteMessage, Kafka client requests all Topics from the Cluster, then filter those by the given Message topic. This is making the cluster to log attempts of topic creation, thus flooding the cluster. This was identified by the client's user issuing the request.
Question: Is there a way to prevent the Client from requesting those Topics ? Instead just using the Topic which is provided by the Message or the Writer ?
Kafka Version 2.6.1
To Reproduce Have a cluster with many topics. Have info logging of this cluster. Publish a message to a specific topic. Check the cluster logging interaction on not owned topic by the previous issued request
Expected behavior Client should not interact with topics which it doesn't own, or request is not target at.
Additional context
Hi @wesleyvicthor I'll be looking into this, as it's possibly related to the other issue I'm investigating (#683).
Hi @dtjm , does this PR https://github.com/segmentio/kafka-go/pull/700 solve this behaviour ?
@WaldoFR it does for this particular repro case, but it causes other tests to fail, so I'm afraid it's probably introducing some regressions. I've gotten deep in the weeds of making sure I didn't break anything else, but you are welcome to test the branch and see if it works for your case.