[improve][broker]PIP-255 Add topic metrics for the number of production data requests to add a topic and the average number of messages per request.

[yyj8]

Motivation

In the current topic level monitoring, there is no metrics to monitor the number of send data requests and the number of messages per send request (i.e., each entry) for a topic. This is inconvenient for us to troubleshoot the following issues:

1. In cases where the client does not perform batch data writing or the batch written messages are not very reasonable, we cannot intuitively reflect through monitoring and cannot detect unreasonable business usage in a timely manner; It has a significant impact on production and consumption performance, increasing cluster load, reducing cluster performance, and increasing overall costs;

2. When we enable the production flow limiting strategy, if the flow limiting condition is triggered by the number of requests on a single connection, it is difficult to accurately locate which topic's high-frequency send data is causing it;

Modifications

Add four metrics: pulsar_request_rate_in: Topic send request rate. pulsar_average_msg_per_request: Average number of messages per send request for topic.

pulsar_producer_request_in: producer send request rate. pulsar_producer_avg_msg_per_request: Average number of messages per send request for producer.

Does this pull request potentially affect one of the following parts:

If the box was checked, please highlight the changes

• [ ] Dependencies (add or upgrade a dependency)
• [ ] The public API
• [ ] The schema
• [ ] The default values of configurations
• [ ] The threading model
• [ ] The binary protocol
• [ ] The REST endpoints
• [ ] The admin CLI options
• [x] The metrics
• [ ] Anything that affects deployment

Documentation

• [ ] doc
• [x] doc-required
• [ ] doc-not-needed
• [ ] doc-complete

Matching PR in forked repository

PR in forked repository: https://github.com/yyj8/pulsar/pull/6

[yyj8]

/pulsarbot run-failure-checks

[github-actions[bot]]

The pr had no activity for 30 days, mark with Stale label.

[yyj8]

Did this PIP vote pass? Where is the issue?

PIP-255 is Make the partition assignment strategy pluggable #19806

PIP must have an issue, right？

[315157973]

Did this PIP vote pass? Where is the issue? PIP-255 is Make the partition assignment strategy pluggable #19806

PIP must have an issue, right？

pip needs to be discussed in the dev mail list, and finally initiate a vote, at least 2 PMC +1 can pass

[yyj8]

Did this PIP vote pass? Where is the issue? PIP-255 is Make the partition assignment strategy pluggable #19806

PIP must have an issue, right？

pip needs to be discussed in the dev mail list, and finally initiate a vote, at least 2 PMC +1 can pass

Do I still need to send an email to the development mailing list？

[github-actions[bot]]

The pr had no activity for 30 days, mark with Stale label.

[yyj8]

Did this PIP vote pass? Where is the issue? PIP-255 is Make the partition assignment strategy pluggable #19806

PIP must have an issue, right？

pip needs to be discussed in the dev mail list, and finally initiate a vote, at least 2 PMC +1 can pass

mail list：https://lists.apache.org/thread/w4jrs57gx69xw28psn0lnzxxtxnmrllo