node_exporter icon indicating copy to clipboard operation
node_exporter copied to clipboard

avg RTT in mountstats metrics

Open sreejith-vk opened this issue 2 years ago • 7 comments

Host operating system: output of uname -a

uname -a

Linux<> 4.12.14-150100.197.117-default #1 SMP Sat Jul 9 11:46:36 UTC 2022 (8d500b6) x86_64 x86_64 x86_64 GNU/Linux

mountstats metrics provide a lot of statistics. The statistics can only be guessed based on the name. I could not find any documentation on the metric and how the statistics can be interpreted.

For example, the nfsiostat command provides me following data :

 nfsiostat 1  1 <mountpoint>

<mountdevice> mounted on <mountpoint>

           ops/s       rpc bklog
          12.140           0.000

read:              ops/s            kB/s           kB/op         retrans    avg RTT (ms)   avg exe (ms)
                   0.687         135.679         197.518        0 (0.0%)          9.235          9.876
write:             ops/s            kB/s           kB/op         retrans    avg RTT (ms)    avg exe (ms)
                   1.713          67.570          39.446        0 (0.0%)           3.700           3.767

I am trying to figure out if avg RTT can be analysed using node exporter.

avg RTT (ms) - This is the duration from the time that client's kernel sends the RPC request until the time it receives the reply.

When I look at mountstats metrics I could find a few metrics related:

node_mountstats_nfs_operations_major_timeouts_total{export="<MASKED>",mountaddr="<MASKED>",operation="READ",protocol="tcp"} 0
node_mountstats_nfs_operations_queue_time_seconds_total{export="MASKED",mountaddr="MASKED",operation="READ",protocol="tcp"} 120.876
node_mountstats_nfs_operations_request_time_seconds_total{export="MASKED",mountaddr="MASKED",operation="READ",protocol="tcp"} 1935.914
node_mountstats_nfs_operations_response_time_seconds_total{export="MASKED",mountaddr="MASKED",operation="READ",protocol="tcp"} 1810.259

Do we already have avg RTT? As this is handy information for troubleshooting, can we add this information if it's feasible?

sreejith-vk avatar Dec 15 '22 14:12 sreejith-vk

Seems like a reasonable request. As usual, parsing should go into procfs.

discordianfish avatar Dec 17 '22 14:12 discordianfish

Hi @discordianfish, Hope you're well! Can I send a PR for this? Will you be able to review?

Thanks!

ksankeerth avatar Dec 29 '22 12:12 ksankeerth

@ksankeerth yes! Feel free to assign the PR to me

discordianfish avatar Jan 05 '23 13:01 discordianfish

@ksankeerth yes! Feel free to assign the PR to me

I opened a PR in procfs. https://github.com/prometheus/procfs/pull/487 Please check. Once it's merged, I'll send a PR to node_exporter.

Thanks

ksankeerth avatar Jan 11 '23 12:01 ksankeerth

Hi, can I check what is the progress on enabling the nfsiostats metric such as retrans, avg RTT and avg exe in node exporter please?

KelvinTanJF avatar May 11 '23 01:05 KelvinTanJF

@KelvinTanJF, PR for procfs got merged recently https://github.com/prometheus/procfs/pull/487. I think, PR for this node_exporter can be merged after review.

ksankeerth avatar May 11 '23 02:05 ksankeerth

Hi @ksankeerth, noted with thanks. Can I check with you what is the targeted release date please?

KelvinTanJF avatar May 11 '23 03:05 KelvinTanJF