Add Prometheus Layer support so that users can use prometheus client instead

[Xuanwo]

[flaneur2020]

can we pass a registry parameter to the prometheus layer? so it can easier to integrate with the caller's prometheus config.

[Xuanwo]

can we pass a registry parameter to the prometheus layer? so it can easier to integrate with the caller's prometheus config.

Seems interesting.

I know little about rust-prometheus's API now so I don't know if we can. But I will take this into consideration.

[Xuanwo]

After a quick look on rust-prometheus docs, I found allowing user-input registry is required.

We will implement it. @flaneur2020

[wcy-fdu]

Is this issue work in progress? RisingWave used to implement it's monitoring system 1.0 via rust-prometheus and grafana dashboard, which include some basic metrics for storage. If no one is assigned , I'd like to try it.

[Xuanwo]

Go ahead!

[wcy-fdu]

Can you describe the requirements in detail? I saw that there are already MetricsHandler in layers, do you want users to be able to observe these metrics or add more metrics? For observation, I prefer to introduce grafana dashboard to monitor these metrics, just like Tikv or RisingWave; for more detailed metrics, may add some metrics for the basic operations on object, eg. op_latency, op_count, object_size.

[Xuanwo]

Can you describe the requirements in detail?

PrometheusLayer should be exactly the same with MetricsLayer, the only difference is all metrics are sent via promethues handler.

I saw that there are already MetricsHandler in layers, do you want users to be able to observe these metrics or add more metrics?

PrometheusLayer and MetricsLayer are different layers that users can enable one of them or at the same time.

For observation, I prefer to introduce grafana dashboard to monitor these metrics, just like Tikv or RisingWave;

Great! PrometheusLayer or MetricsLayer are built for this. It may be a good idea to include a grafana template json in our docs for our users to get started.

for more detailed metrics, may add some metrics for the basic operations on object, eg. op_latency, op_count, object_size.

Latency and count are already covered. And we have opendal_bytes_total for read & write, maybe that enough?

[Xuanwo]

Thanks for @wcy-fdu 's work, this issue has been implemented.