Support OSD operations Latency for Nautilus release

[homer314]

Hi guys, at the moment, ceph_exporter handle commit & apply latency metrics (pretty he same thing, for bluestore). In Nautilus there are other operations latency related metrics, really useful to watch. Those metrics are op_*_latency, for example:

• ceph_osd_op_latency_count
• ceph_osd_op_latency_sum
• ceph_osd_op_r_latency_count
• ceph_osd_op_r_latency_sum
• ceph_osd_op_rw_latency_count
• ceph_osd_op_rw_latency_sum
• ceph_osd_op_w_latency_count
• ceph_osd_op_w_latency_sum

As i wrote, it would be really, really usefull to have those metrics colleted. Is it possibile to update collectors?

Regards Flavio

[alram]

Hi Flavio, I tried to research this for a bit but couldn't find much info.

I believe these are from the mgr prometheus module? (https://docs.ceph.com/en/latest/mgr/prometheus), is this correct?

From reading their name, it sounds like these are metrics coming from the OSD asok directly as opposed to commit/apply latency that comes from the mon. ceph_exporter doesn't have the ability to extract data from Ceph's admin socket and this is not something we intend to add to ceph_exporter

Let me know if I'm correct about the metrics sources, if not please point me to the commands that generate these stats and happy to have a look at how we can implement them.

[homer314]

Hi Alexandre, you're right, those comes from prometheus module. I was thinking this exporter acts as a prometheus module (broker?) client/exporter at the same time.

Thank you Flavio

[alram]

ceph_exporter sadly cannot communicate with OSDs' asok. I don't think it would be architecturally possible for us to do that from scratch as it would require access (ssh or otherwise) from the ceph_exporter instance to every OSD host. The only straightforward to do it would be to enable the ceph-mgr prometheus module and get the data from it into ceph_exporter but since the ceph-mgr prom module already as most if not all the info that ceph_exporter provides it would be redundant to run both so I don't think we are going to add that data

[alram]

Closing since we've no plan to add this