pulsar icon indicating copy to clipboard operation
pulsar copied to clipboard

[improve][broker] allow client skips the non-continued deleted ranges stats

Open leizhiyuan opened this issue 3 years ago • 3 comments

Fixes #18240

Master Issue: #18240

Motivation

Modifications

add a flag getTotalNonContiguousDeletedMessagesRange to allow users can skip the non-continued deleted ranges stats

Verifying this change

  • [ ] Make sure that the change passes the CI checks.

https://github.com/leizhiyuan/pulsar/pull/4

(Please pick either of the following options)

This change is a trivial rework / code cleanup without any test coverage.

(or)

This change is already covered by existing tests, such as (please describe tests).

(or)

This change added tests and can be verified as follows:

(example:)

  • Added integration tests for end-to-end deployment with large payloads (10MB)
  • Extended integration test for recovery after broker failure

Does this pull request potentially affect one of the following parts:

If the box was checked, please highlight the changes

  • [ ] Dependencies (add or upgrade a dependency)
  • [ ] The public API
  • [ ] The schema
  • [ ] The default values of configurations
  • [ ] The threading model
  • [ ] The binary protocol
  • [x] The REST endpoints
  • [ ] The admin CLI options
  • [ ] Anything that affects deployment

Documentation

  • [ ] doc
  • [ ] doc-required
  • [x] doc-not-needed
  • [ ] doc-complete

Matching PR in forked repository

PR in forked repository:

leizhiyuan avatar Oct 31 '22 05:10 leizhiyuan

maybe we should use GetStatsOptions next pr, not original name like getPreciseBacklog,subscriptionBacklogSize etc.

leizhiyuan avatar Oct 31 '22 05:10 leizhiyuan

@leizhiyuan Please add the following content to your PR description and select a checkbox:

- [ ] `doc` <!-- Your PR contains doc changes -->
- [ ] `doc-required` <!-- Your PR changes impact docs and you will update later -->
- [ ] `doc-not-needed` <!-- Your PR changes do not impact docs -->
- [ ] `doc-complete` <!-- Docs have been already added -->

github-actions[bot] avatar Oct 31 '22 05:10 github-actions[bot]

There is a failed broker test in your https://github.com/leizhiyuan/pulsar/pull/4 : https://github.com/leizhiyuan/pulsar/actions/runs/3373856231/jobs/5598944551#step:10:1209

if it is caused by your change please fix it.

labuladong avatar Nov 02 '22 04:11 labuladong

There is a failed broker test in your leizhiyuan#4 : https://github.com/leizhiyuan/pulsar/actions/runs/3373856231/jobs/5598944551#step:10:1209

if it is caused by your change please fix it.

https://github.com/leizhiyuan/pulsar/pull/5

I didn't sync the master branch ,now I opened a new pr

leizhiyuan avatar Nov 06 '22 00:11 leizhiyuan

There is a failed broker test in your leizhiyuan#4 : https://github.com/leizhiyuan/pulsar/actions/runs/3373856231/jobs/5598944551#step:10:1209 if it is caused by your change please fix it.

leizhiyuan#5

I didn't sync the master branch ,now I opened a new pr

image

Add some case seems be flaky, It is not related with the pr, I ran ok in my machine。

leizhiyuan avatar Nov 06 '22 00:11 leizhiyuan

/pulsarbot run-failure-checks

codelipenghui avatar Nov 16 '22 01:11 codelipenghui

Codecov Report

Merging #18263 (8460be2) into master (46c0438) will increase coverage by 5.87%. The diff coverage is 83.01%.

Impacted file tree graph

@@             Coverage Diff              @@
##             master   #18263      +/-   ##
============================================
+ Coverage     38.84%   44.71%   +5.87%     
- Complexity     8286    10591    +2305     
============================================
  Files           683      748      +65     
  Lines         67323    73915    +6592     
  Branches       7215     8167     +952     
============================================
+ Hits          26152    33052    +6900     
+ Misses        38210    37169    -1041     
- Partials       2961     3694     +733     
Flag Coverage Δ
unittests 44.71% <83.01%> (+5.87%) :arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...he/pulsar/broker/admin/v2/NonPersistentTopics.java 62.03% <ø> (ø)
...rg/apache/pulsar/broker/service/BrokerService.java 57.11% <0.00%> (-0.81%) :arrow_down:
...n/java/org/apache/pulsar/broker/service/Topic.java 18.18% <ø> (ø)
...ker/stats/prometheus/NamespaceStatsAggregator.java 0.00% <0.00%> (ø)
...pulsar/broker/admin/impl/PersistentTopicsBase.java 59.53% <81.81%> (+6.96%) :arrow_up:
...pache/pulsar/broker/admin/v1/PersistentTopics.java 46.49% <100.00%> (+0.79%) :arrow_up:
...pache/pulsar/broker/admin/v2/PersistentTopics.java 75.54% <100.00%> (+4.82%) :arrow_up:
...sar/broker/resourcegroup/ResourceGroupService.java 38.10% <100.00%> (+0.33%) :arrow_up:
...oker/service/nonpersistent/NonPersistentTopic.java 57.46% <100.00%> (+2.57%) :arrow_up:
...ker/service/persistent/PersistentSubscription.java 68.32% <100.00%> (+3.90%) :arrow_up:
... and 304 more

codecov-commenter avatar Nov 16 '22 02:11 codecov-commenter

The pr had no activity for 30 days, mark with Stale label.

github-actions[bot] avatar Dec 22 '22 01:12 github-actions[bot]

@leizhiyuan could you please merge the apache/master, then rerun the test

congbobo184 avatar Dec 24 '22 08:12 congbobo184

The pr had no activity for 30 days, mark with Stale label.

github-actions[bot] avatar Jan 24 '23 02:01 github-actions[bot]

@leizhiyuan could you please rebase master and take a look at the failed tests?

poorbarcode avatar Apr 10 '23 16:04 poorbarcode

The pr had no activity for 30 days, mark with Stale label.

github-actions[bot] avatar May 11 '23 01:05 github-actions[bot]