pulsar icon indicating copy to clipboard operation
pulsar copied to clipboard

[feat][broker] Support dynamic update log level at runtime

Open ethqunzhong opened this issue 3 years ago • 8 comments

Motivation

As default, Broker log level set by log4j2.yaml with pulsar.log.level=info, if developers want to update log level to debug, need to Modify log42j.yaml#pulsar.log.level=debug and then restart Broker service.

This PR support developers dynamic update Broker log level at runtime,It is very useful for online trouble shooting without having to stop Broker service and start it again.

Modifications

  • add admin cli (bin/pulsar-admin brokers update-logger-level --classname ${CLASSNAME} --level ${LEVEL})
  • add rest endpoints (/admin/v2/brokers/log4j/{classname}/{level})

Verifying this change

  • Make sure that the change passes the CI checks.
  • This change added tests and can be verified as follows:org.apache.pulsar.admin.cli.PulsarAdminToolTest#brokers
  • works as expected on product environment.

Does this pull request potentially affect one of the following parts:

If yes was chosen, please highlight the changes

  • Dependencies (does it add or upgrade a dependency): (no)
  • The public API: (no)
  • The schema: (no)
  • The default values of configurations: (no)
  • The wire protocol: (no)
  • The rest endpoints: (yes)
  • The admin cli options: (yes)
  • Anything that affects deployment: (no)

Documentation

  • [x] doc-not-needed

ethqunzhong avatar Feb 10 '22 03:02 ethqunzhong

cc @eolivelli @Jason918 PTAL again thanks

wolfstudy avatar Feb 21 '22 09:02 wolfstudy

/pulsarbot run-failure-checks

ethqunzhong avatar Feb 22 '22 07:02 ethqunzhong

IIRC log4j supports automatic reloading of configuration files. Isn't it enough for you?

yes, Log4j has the ability to automatically configure itself during initialization and reload config while config files changed. but it is more convenient for developers update logger level by programmatically way.

ethqunzhong avatar Feb 22 '22 10:02 ethqunzhong

The pr had no activity for 30 days, mark with Stale label.

github-actions[bot] avatar Apr 21 '22 02:04 github-actions[bot]

The pr had no activity for 30 days, mark with Stale label.

github-actions[bot] avatar May 28 '22 02:05 github-actions[bot]

@ethqunzhong Please add the following content to your PR description and select a checkbox:

- [ ] `doc` <!-- Your PR contains doc changes -->
- [ ] `doc-required` <!-- Your PR changes impact docs and you will update later -->
- [ ] `doc-not-needed` <!-- Your PR changes do not impact docs -->
- [ ] `doc-complete` <!-- Docs have been already added -->

github-actions[bot] avatar Dec 15 '22 23:12 github-actions[bot]

Codecov Report

Merging #14200 (c94489f) into master (3011946) will increase coverage by 0.26%. The diff coverage is 0.00%.

Impacted file tree graph

@@             Coverage Diff              @@
##             master   #14200      +/-   ##
============================================
+ Coverage     46.75%   47.01%   +0.26%     
+ Complexity    10514     9323    -1191     
============================================
  Files           703      623      -80     
  Lines         68858    59132    -9726     
  Branches       7383     6148    -1235     
============================================
- Hits          32193    27800    -4393     
+ Misses        33065    28309    -4756     
+ Partials       3600     3023     -577     
Flag Coverage Δ
unittests 47.01% <0.00%> (+0.26%) :arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...g/apache/pulsar/broker/admin/impl/BrokersBase.java 55.28% <0.00%> (-6.54%) :arrow_down:
.../pulsar/broker/service/SharedConsumerAssignor.java 3.70% <0.00%> (-74.08%) :arrow_down:
...java/org/apache/pulsar/proxy/stats/TopicStats.java 58.82% <0.00%> (-41.18%) :arrow_down:
...apache/pulsar/broker/service/EntryAndMetadata.java 0.00% <0.00%> (-40.75%) :arrow_down:
...ersistentStickyKeyDispatcherMultipleConsumers.java 46.07% <0.00%> (-14.71%) :arrow_down:
.../apache/pulsar/broker/admin/impl/PackagesBase.java 54.12% <0.00%> (-13.77%) :arrow_down:
...ervice/persistent/MessageRedeliveryController.java 56.09% <0.00%> (-12.20%) :arrow_down:
...e/pulsar/broker/service/EntryBatchIndexesAcks.java 82.14% <0.00%> (-10.72%) :arrow_down:
...sistent/PersistentDispatcherMultipleConsumers.java 50.34% <0.00%> (-8.51%) :arrow_down:
...balance/impl/SimpleResourceAllocationPolicies.java 48.57% <0.00%> (-5.72%) :arrow_down:
... and 108 more

codecov-commenter avatar Dec 16 '22 00:12 codecov-commenter

Since we will start the RC version of 3.0.0 on 2023-04-11, I will change the label/milestone of PR who have not been merged.

  • The PR of type feature is deferred to 3.1.0
  • The PR of type fix is deferred to 3.0.1

So drag this PR to 3.1.0

poorbarcode avatar Apr 10 '23 15:04 poorbarcode