Added support for exposing Stream Connection metrics

[markus812498]

Proposed Changes

This PR adds support for metrics from the rabbit_stream_consumer_created and rabbit_stream_publisher_created ETS tables to be exposed through prometheus through more user-friendly named metrics endpoints. Based off the conversation in a previous PR: https://github.com/rabbitmq/rabbitmq-server/pull/3043

The change is requested to ensure customers using stream connections can scrape metrics from the prometheus endpoints.

Types of Changes

What types of changes does your code introduce to this project? Put an x in the boxes that apply

• [ ] Bug fix (non-breaking change which fixes issue #NNNN)
• [x] New feature (non-breaking change which adds functionality)
• [ ] Breaking change (fix or feature that would cause an observable behavior change in existing systems)
• [ ] Documentation improvements (corrections, new content, etc)
• [ ] Cosmetic change (whitespace, formatting, etc)
• [ ] Build system and/or CI

Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask on the mailing list. We're here to help! This is simply a reminder of what we are going to look for before merging your code.

• [x] I have read the CONTRIBUTING.md document
• [x] I have signed the CA (see https://cla.pivotal.io/sign/rabbitmq)
• [ ] I have added tests that prove my fix is effective or that my feature works
• [ ] All tests pass locally with my changes
• [ ] If relevant, I have added necessary documentation to https://github.com/rabbitmq/rabbitmq-website
• [ ] If relevant, I have added this change to the first version(s) in release-notes that I expect to introduce it

Further Comments

If PR is accepted I think this should also be pushed to the documentation on the rabbitmq-website. Do I create a PR to that repo once this is merged? Or is there another procedure to follow?

Thanks!

[michaelklishin]

This has conflicts with main and needs rebasing.

[gomoripeti]

there are valid test failures. We are working on fixing them...

[markus812498]

after @gomoripeti 's push above, I believe this PR is ready for review as all Prometheus related tests passing and conflicts are resolved. (looks like failing tests are related to MQTT_V5)

[luos]

Hi!

This looks very good, thank you for implementing it.

What do you think of the idea to expose the segments count as well for a stream, then the rough stream size could be calculated from max-segment-size * segments? (Maybe in a separate PR.)

[michaelklishin]

Exposing the current segment count sounds OK to me.

[markus812498]

that sounds good! perhaps I could look into that in a different PR once this gets merged :)

[gomoripeti]

force-push was a rebase to latest main. google-github-actions/auth fails in CI

[michaelklishin]

CI fails because external PRs do not have access to the necessary secrets. We will run the tests locally.

[michaelklishin]

Indeed, https://github.com/cloudamqp/rabbitmq-server/ still uses master as its default branch and https://github.com/cloudamqp/rabbitmq-server/tree/main specifically is from May 15th.

[michaelklishin]

Rebasing cloudamqp/rabbitmq-server@main onto rabbitmq/rabbitmq-server@main is a fast forward.

Then rebasing this PR onto the result will conflict with https://github.com/rabbitmq/rabbitmq-server/pull/11431 but that should be it.

[gomoripeti]

I used the github UI button "Rebase" previously, sorry if it caused some trouble.

Now I manually rebased to latest main @ rabbitmq/rabbitmq-server and fixed the conflict with #11431