kafka icon indicating copy to clipboard operation
kafka copied to clipboard

KAFKA-19940 Reduce the log4j2 cpu usage in PersisterStateManager

Open m1a2st opened this issue 1 month ago • 3 comments

FYI: https://github.com/apache/kafka/pull/20175/files#r2572693459

m1a2st avatar Dec 01 '25 13:12 m1a2st

Hi @m1a2st , Thanks for the PR. While the code is valid, it would mutate the logger key for the internal handlers to

org.apache.kafka.server.share.persister.PersisterStateManager$WriteStateHandler

respectively

This would make it infeasible to turn on DEBUG, TRACE logging for these classes. For example on a local kafka cluster, these fail:

bin/kafka-configs.sh --bootstrap-server localhost:9092 --alter --add-config "org.apache.kafka.server.share.persister.PersisterStateManager$WriteStateHandler=DEBUG" --entity-type broker-loggers --entity-name 1
Invalid broker logger(s): org.apache.kafka.server.share.persister.PersisterStateManager
bin/kafka-configs.sh --bootstrap-server localhost:9092 --alter --add-config "org.apache.kafka.server.share.persister.PersisterStateManager\$WriteStateHandler=DEBUG" --entity-type broker-loggers --entity-name 1
Invalid character found for config key: org.apache.kafka.server.share.persister.PersisterStateManager$WriteStateHandler

This was the reason why original code used canonical name, where possible.

cc: @apoorvmittal10

smjn avatar Dec 05 '25 14:12 smjn

Thanks for @smjn comment, I have updated to use the getCanonicalName(), and I did the following step for testing

  1. Create a local Kafka cluster.
  2. Produce and Consumer some messge
bin/kafka-console-producer.sh --topic test --bootstrap-server localhost:9092
>4
>5
>6
./bin/kafka-console-share-consumer.sh --bootstrap-server localhost:9092 --topic test --group test
4
5
6
  1. Alfer the org.apache.kafka.server.share.persister.PersisterStateManager.WriteStateHandler log level
bin/kafka-figs.sh --bootstrap-server localhost:9092 --alter --add-config "org.apache.kafka.server.share.persister.PersisterStateManager.WriteStateHandler=DEBUG" --entity-type broker-loggers --entity-name 1
Completed updating config for broker-logger 1.

m1a2st avatar Dec 06 '25 05:12 m1a2st

This was the reason why original code used canonical name, where possible.

I think that is a bug in ConfigCommand. Many log names include the $ symbol, and that is completely legal when using Admin#incrementalAlterConfigs

It seems to me using LoggerFactory.getLogger(DefaultStatePersister.class) is good and simple, and then we could fix ConfigCommand in https://issues.apache.org/jira/browse/KAFKA-19980

chia7712 avatar Dec 10 '25 03:12 chia7712

It seems to me using LoggerFactory.getLogger(DefaultStatePersister.class) is good and simple, and then we could fix ConfigCommand in https://issues.apache.org/jira/browse/KAFKA-19980

@smjn What do you think about this comment? Your feedback would be conducive before I merge the changes

chia7712 avatar Dec 12 '25 06:12 chia7712