DEVX-2610: Replace ccloud-exporter container/scrape/dashboard with OpenMetrics /export scrape

[javabrett]

Description

https://confluentinc.atlassian.net/browse/DEVX-2610 https://confluentinc.atlassian.net/browse/DEVX-2732 https://confluentinc.atlassian.net/browse/DEVX-2733

What behavior does this PR change, and why?

Replace ccloud-exporter container/scrape/dashboard with OpenMetrics /export scrape. This removes the need for the shim provided by the ccloud-exporter container. This PR replaces that scrape job with the equivalent direct scape of /export, and makes the required Grafana dashboard changes.

This PR also adds detection and configuration for Connectors and ksqlDB metrics, should those be added to the stack - they are not added as resources in the base demo, but see note below re ksqlDB.

Also updates Grafana and Prometheus images to their latest/current versions.

Note

• This demo does not create any sample connectors, so that part of the dashboard will always remain empty.
• It was noticed that the command confluent ksql cluster list has environment-scope by default (reported this in case it is a bug), whereas the equivalent for connectors is cluster-scope (expected). I just let this go, but the outcome is that if you have another cluster in the same environment with a ksqlDB cluster running, this demo, which doesn't create ksqlDB clusters, will still detect and display the single-current metric for it (CSUs).

Author Validation

Describe the validation already done, or needs to be done, by the PR submitter.

• [X] Documentation

• [X] ccloud-observability

Reviewer Tasks

Describe the tasks/validation that the PR submitter is requesting to be done by the reviewer.

• [ ] Documentation

• [ ] ccloud-observability

[javabrett]

Testing for this could probably benefit from having https://github.com/confluentinc/examples/pull/1089 merged first.

[sirianni]

@sirianni can you please confirm whether you reviewed the PR or whether you validated the code (did you run the entire example)?

I did not run the example. I sanity checked the code visually and didn't see anything that was obviously incorrect.

[ybyzek]

@sirianni can you please confirm whether you reviewed the PR or whether you validated the code (did you run the entire example)?

I did not run the example. I sanity checked the code visually and didn't see anything that was obviously incorrect.

@javabrett it would be good to have a 2nd person run the example

[ybyzek]

cc: @mayur-mwar may have time to validate this next week