pulsar icon indicating copy to clipboard operation
pulsar copied to clipboard

[fix][txn]Make txn metrics name conforms to the rule

Open poorbarcode opened this issue 3 years ago • 2 comments

Fixes: #17921

Note:

This patch will change metrics name s_bufferedwriter_batch_record_count and s_bufferedwriter_batch_oldest_record_delay_time_second. These two names were first used in this PR #17701, and will be released in version 2.11.1, so this change will not cause any breaking changes.

Motivation

https://github.com/poorbarcode/pulsar/actions/runs/3156649582/jobs/5136584463 https://github.com/apache/pulsar/actions/runs/3156649597/jobs/5136596447

Problem-1

If the Prometheus-Colloctor which typed Counter is named 'xxx_count', then the output metrics-api will be named 'xxx_count_count'.

TxnLogBufferedWriterMetricsStats hits this error.

https://github.com/apache/pulsar/blob/fb7307d8f4998e42b18df3a4599fd7ec34cb04a9/pulsar-transaction/coordinator/src/main/java/org/apache/pulsar/transaction/coordinator/impl/TxnLogBufferedWriterMetricsStats.java#L105-L106


Problem-2

PrometheusMetricsTest defines the standard metrics name(see code below):

["_sum", "_bucket", "_count", "_total", "_created"]

But the standard Prometheus name has three others( see: https://github.com/prometheus/client_java/blob/c28b901225e35e7c1df0eacae8b58fdfbb390162/simpleclient/src/main/java/io/prometheus/client/Collector.java#L152-L186 ):

["_info", "_gsum", "_gcount"]

https://github.com/apache/pulsar/blob/fb7307d8f4998e42b18df3a4599fd7ec34cb04a9/pulsar-broker/src/test/java/org/apache/pulsar/broker/stats/PrometheusMetricsTest.java#L834-L861


Modifications

  • Make PrometheusMetricsTest run with transaction feature
  • Make txn metrics name conforms to the rule. see: https://prometheus.io/docs/practices/naming/
  • Make PrometheusMetricsTest support all suffix of prometheus metrics name

Documentation

  • [x] doc-not-needed (Please explain why)

Matching PR in forked repository

PR in forked repository:

  • https://github.com/poorbarcode/pulsar/pull/19

poorbarcode avatar Sep 30 '22 06:09 poorbarcode

@poorbarcode Please help confirm that we don't have a release for the metrics name that you want change.

codelipenghui avatar Oct 08 '22 02:10 codelipenghui

Hi @codelipenghui

Yes, we don't have a release for these metrics names that I want to change. Thanks

poorbarcode avatar Oct 08 '22 03:10 poorbarcode

These two names were first used in this PR https://github.com/apache/pulsar/pull/17701, and will be released in version 2.11.1

We will not release 17701 in 2.11.1 right?

codelipenghui avatar Nov 11 '22 11:11 codelipenghui

/pulsarbot run-failure-checks

codelipenghui avatar Nov 11 '22 11:11 codelipenghui

@codelipenghui

We will not release 17701 in 2.11.1 right?

Correct, PR #17701 hasn't cherry-picked any branches yet

poorbarcode avatar Nov 11 '22 11:11 poorbarcode

Codecov Report

Merging #17905 (55dc2c0) into master (79a97a9) will decrease coverage by 0.04%. The diff coverage is n/a.

Impacted file tree graph

@@             Coverage Diff              @@
##             master   #17905      +/-   ##
============================================
- Coverage     47.24%   47.20%   -0.05%     
+ Complexity    10407    10405       -2     
============================================
  Files           692      692              
  Lines         67766    67766              
  Branches       7258     7258              
============================================
- Hits          32016    31988      -28     
- Misses        32158    32193      +35     
+ Partials       3592     3585       -7     
Flag Coverage Δ
unittests 47.20% <ø> (-0.05%) :arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...java/org/apache/pulsar/proxy/stats/TopicStats.java 58.82% <0.00%> (-41.18%) :arrow_down:
.../pulsar/broker/service/BrokerServiceException.java 42.59% <0.00%> (-5.56%) :arrow_down:
...rg/apache/pulsar/broker/service/AbstractTopic.java 61.71% <0.00%> (-3.16%) :arrow_down:
...oker/service/schema/SchemaRegistryServiceImpl.java 58.85% <0.00%> (-1.81%) :arrow_down:
...g/apache/pulsar/broker/lookup/TopicLookupBase.java 52.51% <0.00%> (-1.68%) :arrow_down:
...he/pulsar/broker/admin/v2/NonPersistentTopics.java 60.64% <0.00%> (-1.39%) :arrow_down:
...rg/apache/pulsar/proxy/server/ProxyConnection.java 55.21% <0.00%> (-1.13%) :arrow_down:
...rg/apache/pulsar/broker/web/PulsarWebResource.java 57.20% <0.00%> (-1.09%) :arrow_down:
...a/org/apache/pulsar/proxy/server/ProxyService.java 80.00% <0.00%> (-0.94%) :arrow_down:
...sistent/PersistentDispatcherMultipleConsumers.java 57.52% <0.00%> (-0.89%) :arrow_down:
... and 27 more

codecov-commenter avatar Nov 11 '22 12:11 codecov-commenter