kafka icon indicating copy to clipboard operation
kafka copied to clipboard

One slow broker can slow fetch responses from all brokers

Open thomaslee opened this issue 7 years ago • 0 comments

The call to Client.fetchRequest in BaseConsumer_fetch wraps requests to all brokers in a Promise.all(), meaning that one slow/misbehaving broker can leave us idle in spite of data being returned by all other brokers.

Returning a Promise.map(...) might be better, but would probably require changes to what is returned by fetchRequest (from a single topics object to individual topic objects per leader) & I'm not entirely sure what guarantees we're making about API stability.

Locally I've tinkered with splitting up calls to fetchRequest by leader in BaseConsumer order to fully decouple broker requests from one another & avoid this issue without tweaking Client semantics, but that seems a bit dirty.

thomaslee avatar Jan 30 '18 09:01 thomaslee