jmeter-prometheus-plugin

jmeter-prometheus-plugin copied to clipboard

Hi,

Good Day !

When I compared Response Time data in Premethues with jmeter jtl file, P90,P95 , Min and Max are not matching and they are 2 to 3 times higher than the data we have in jtl file.

I am using Summary metric to write Response time data into Premethues and using following quantiles: 0.0,0.5|0.90,0.01|0.95,0.01 | 1.0,0.01

Can you please let know anything I am missing in the configuration / resolution for this issue ?

Thanks in advance, Ramu

I would guess the issue is this: Aggregate Report (that's what you're using to check against?) is using the entire dataset to calculate against. These metrics are only using the last 10 minutes of data to calculate against.

From these docs: Optionally, after a ; separator the length of the window used to calculate the quantile can be specified.

However, you can change this to say 60 minutes (if you're running a test for an hour) by adding ;60 to the end of each bucket.

0.0,0.5;60|0.90,0.01;60|0.95,0.01;60|1.0,0.01;60

Thanks a lot for you reply. Though I have added additional quantiles and length of time window for Response Time Summary metric, its taking only default values.

content from Jmeter Script:

0,0.1;15|0.90,0.1;15|0.95,0.1;15|1.0,0.01;15 ResponseTime

Content from scraper:

JM_ResponseTime_Summary{label="/contacts.php",quantile="0.75",} 299.0 JM_ResponseTime_Summary{label="/contacts.php",quantile="0.95",} 508.0 JM_ResponseTime_Summary{label="/contacts.php",quantile="0.99",} 899.0

Jar version I am using for testing is -> jmeter-prometheus-plugin-0.6.0.jar Can you please look into it ?

Regards, Ramu

Can you turn debug logging on and see if there's any issues it's printing?

Are you saying that if you have a configuration like this it does work? (i.e., without changing the window length)

0.90,0.01|0.95,0.01|0.99,0.01

Hi @johrstrom , Thanks a lot for your reply. (1) I did not observed any errors in debug logs, following INFO is printed in the log at the end of the test.

2023-03-14 23:32:11,977 INFO o.a.j.e.StandardJMeterEngine: Notifying test listeners of end of test 2023-03-14 23:32:11,977 DEBUG c.g.j.c.JMeterCollectorRegistry: unregistering JM_ResponseTime_Summary 2023-03-14 23:32:11,977 DEBUG c.g.j.c.JMeterCollectorRegistry: unregistering JM_RequestCount 2023-03-14 23:32:11,977 DEBUG c.g.j.c.JMeterCollectorRegistry: unregistering JM_PassCount 2023-03-14 23:32:11,977 DEBUG c.g.j.c.JMeterCollectorRegistry: unregistering JM_FailCount 2023-03-14 23:32:11,978 INFO o.a.j.g.u.JMeterMenuBar: setRunning(false, local)

(2) config 0.90,0.01|0.95,0.01|0.99,0.01 is working functionally, but value are not matching.

I am using data in Prometheus to prepare a custom summary report once the test is completed and comparing data with Jmeter JTL file for correctness. all the metrics matching except P90,P95, min and max response times. I am using Summary to measure response time.

Please let me know if you need any additional information.

Regards, Ramu

There could be a bug here. I'll have to check to see if there are tests for the same, but cursorily looking at this logic, it doesn't appear to be correct.

https://github.com/johrstrom/jmeter-prometheus-plugin/blob/6109f9f80f37d65d73c0e86ebf46601e3d7e2a4c/src/main/java/com/github/johrstrom/collector/BaseCollectorConfig.java#L423-L444

Try this config. The window length applies to the entire config, so you can only have 1. I've verified the logic and added a test case for the same in #120, so I believe from the test case that already worked on the main line that this should work.

0.90,0.01|0.95,0.01|0.99,0.01;60

I am facing same issue. Above line is not working for me. Any other suggestion to see same values of Aggregate Report on Promethues.

🤦‍♂️ I'm just now realizing that time is in seconds, so 60 is 1 minute, less than the default 10 minutes.

A value of 3600 would be for an hour, if you're running an hour long test.

0.90,0.01|0.95,0.01|0.99,0.01;3600

I am using below in quantiles for 11min window. Can you please help me identify what is wrong as it is still going with default 10 min and also its just showing series for 0.9 and 0.95 only

0.75,0.5|0.9,0.01|0.95,0.01|0.99,0.001;660

Can you please help me identify what is wrong as it is still going with default 10 min and also its just showing series for 0.9 and 0.95 only

There must be a bug somewhere that I'm not seeing in unit tests. In unit tests, this config should work.

I don't even have jmeter on my machines anymore, so I'll have to set it all up and actually fire it up to see what may be breaking.

Hi @johrstrom , Thanks a lot for your reply. (1) I did not observed any errors in debug logs, following INFO is printed in the log at the end of the test.

2023-03-14 23:32:11,977 INFO o.a.j.e.StandardJMeterEngine: Notifying test listeners of end of test 2023-03-14 23:32:11,977 DEBUG c.g.j.c.JMeterCollectorRegistry: unregistering JM_ResponseTime_Summary 2023-03-14 23:32:11,977 DEBUG c.g.j.c.JMeterCollectorRegistry: unregistering JM_RequestCount 2023-03-14 23:32:11,977 DEBUG c.g.j.c.JMeterCollectorRegistry: unregistering JM_PassCount 2023-03-14 23:32:11,977 DEBUG c.g.j.c.JMeterCollectorRegistry: unregistering JM_FailCount 2023-03-14 23:32:11,978 INFO o.a.j.g.u.JMeterMenuBar: setRunning(false, local)

(2) config 0.90,0.01|0.95,0.01|0.99,0.01 is working functionally, but value are not matching.

I am using data in Prometheus to prepare a custom summary report once the test is completed and comparing data with Jmeter JTL file for correctness. all the metrics matching except P90,P95, min and max response times. I am using Summary to measure response time.

Please let me know if you need any additional information.

Regards, Ramu

@vasireddi78 did you find any solution....i have exact usecase

@codegirlpower, apologies for delay in response. no resolution yet on this issue. Did you find any solution ?