kafka icon indicating copy to clipboard operation
kafka copied to clipboard

MINOR: add logs when connection count exceeds configured count

Open showuon opened this issue 3 years ago • 3 comments

Currently, when client tried to connect to broker, and got throttled or exceeding connection count (waiting for available connection slot), there will be no logs output in broker. All the client can see is connection timeout exception. This PR adds logs for this case.

Committer Checklist (excluded from commit message)

  • [ ] Verify design and implementation
  • [ ] Verify test coverage and CI build status
  • [ ] Verify documentation (including upgrade notes)

showuon avatar Aug 26 '22 09:08 showuon

@mimaison @rajinisivaram , please take a look. Thanks.

showuon avatar Aug 26 '22 09:08 showuon

I am not sure about it. In highly loaded system with a lot of connections, this will result in flooding the logs. This is even worst if there are rogue clients. It would be better to rely on metrics to diagnose such issues. I don't remember, do we have metrics for this?

dajac avatar Aug 26 '22 15:08 dajac

@dajac, thanks for the comment. Yes, I agree with you that this could result in flooding the logs in highly loaded system. But still, if the client got blocked due to exceeding max connection limit, it's bad that there's no log in broker side. Expose the info via metrics is a good suggestion. At quick look, I can't find we have any metrics recording this info. I'll check it again later, if no, I'm going to propose a KIP for it.

But back to this PR, it is still not good when this issue happened, but broker keeps silent, even if we have metric's help. Maybe we can log with DEBUG level like @divijvaidya 's suggestion, and user can see this log when troubleshooting issues. WDYT?

showuon avatar Aug 29 '22 09:08 showuon