kafka
kafka copied to clipboard
MINOR: Expose client information on RequestContext as additional public API beyond request logs (continuation of KIP 511)
KIP 511 introduced a ClientInformation class that wraps software (client) name and version and is also set as a property on RequestContext, except unfortunately there's no getter to retrieve this information.
The KIP implemented protocol support, a registry to set it at the network layer per session and integrated with RequestContext, but unfortunately the only "public API" for this information is the broker request logs.
This change exposes client information to custom authorisers as well via RequestConext
, where it can be programatically used in a pluggable fashion as well.
Committer Checklist (excluded from commit message)
- [ ] Verify design and implementation
- [ ] Verify test coverage and CI build status
- [ ] Verify documentation (including upgrade notes)
@methodmissing Thanks for the PR. Could you elaborate a little more on the motivation for this change? Also, don't we need to extend the AuthorizableRequestContext
if you want to get the client information in the plugins?
@methodmissing I forgot to mention that AuthorizableRequestContext
is a public interface so we need a KIP for this change. The process is described here: https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals.
@dajac thanks for the prompt reply. We at Shopify currently have a custom authoriser deployed where we log context like principal
, clientId
, topic
etc. to producer and consumer metrics specific compacted topics to build up insights of our consumer landscape.
We'd also be interested in collecting additional information about the clients given our client landscape (and this is true for most larger organisations) is diverse and vary between Ruby
, Java
, Go
and Python
+ variations of versions amongst those.
Given KIP 511 introduced support for tracking client information per connection and this information is already present in a request context, it could be valuable to also expose it to plugins.
Thanks! I'll draft something
I forgot to mention that AuthorizableRequestContext is a public interface so we need a KIP for this change. The process is described here: https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals.
@methodmissing We also expose a metric with the number of clients per software name and version. Have you already checked this out?
I'm closing this PR as it needs to be discussed in a KIP anyway. Feel free to reopen it if needed.