testkube icon indicating copy to clipboard operation
testkube copied to clipboard

Percentage of failures

Open kfirfer opened this issue 1 year ago • 7 comments

Hello

We sending lots of requests on the test (1M+), and 1 request that fail will consider all test failed, but we tolorate below 1% of failures Would be nice if we could put percentage of failures , if for example set 1%, the test will fail if there is above than 1% failed requests

### Tasks

kfirfer avatar Jul 14 '23 21:07 kfirfer

Thank you, @kfirfer Really excited hearing about 1M requests :) Could you please provide an example of such a test?

vsukhin avatar Jul 14 '23 21:07 vsukhin

@vsukhin its internally in our air-gapped environment not really allowed to take the details =X

kfirfer avatar Jul 14 '23 21:07 kfirfer

Using jmeter exeuter

kfirfer avatar Jul 14 '23 21:07 kfirfer

So, you pass a parameter with amount of repetiion of the test to jmeter and receive response forr your tests 999999 tests passed, 1 test failed

vsukhin avatar Jul 14 '23 21:07 vsukhin

@vsukhin the 1 failed test was example in jmeter we fire 1M of requests sometimes 50 requests failed, sometimes 200 failed etc.. if we put for example 1% threshold, we can have from 1M requests tolerate 999 requests failures and the test will success

kfirfer avatar Jul 14 '23 22:07 kfirfer

thank you @kfirfer let us think about it @TheBrunoLopes @exu @fivenp

vsukhin avatar Jul 14 '23 22:07 vsukhin

Hi @kfirfer great use case, are we somehow able to get this data in normalized way from some report ?

exu avatar Jul 15 '23 04:07 exu