What are the numbers inside parentheses?

[hamin]

Hi there,

I'm trying to figure out what the numbers in parentheses outputted for each stat value in the table mean.

Looking at the code it seems like its happening over here : https://github.com/ionelmc/pytest-benchmark/blob/2ed70632a38909ced0935c010b6696679a28c412/src/pytest_benchmark/table.py#L39

I'm not sure what pos/total means and what I should gleam from those values in the parenthesis.

Also, thank you so much for this tool! It's been a lifesaver so far 😁

[ionelmc]

That line displays stat computation progress indication (computing all those avgs, stddevs and such), so not actual stats.

The values in the results table display relative ratios in the parentheses. Eg:

test_foo  100s 
test_bar  200s (2.0)

Maybe it should output (2.0x) instead for clarity?

[hamin]

@ionelmc Hmm so as an example output (ignore number of rounds and iterations here):

baseline:
---------------------------------------------------------------------------------- benchmark: 2 tests ---------------------------------------------------------------------------------
Name (time in s)             Min                Max               Mean            StdDev             Median               IQR            Outliers     OPS            Rounds  Iterations
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
foo1[50]       2.0069 (1.0)       2.0069 (1.0)       2.0069 (1.0)      0.0000 (1.0)       2.0069 (1.0)      0.0000 (1.0)           0;0  0.4983 (1.0)           1           1
foo2[500]     15.4276 (7.69)     15.4276 (7.69)     15.4276 (7.69)     0.0000 (1.0)      15.4276 (7.69)     0.0000 (1.0)           0;0  0.0648 (0.13)          1           1
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

no-logs:
---------------------------------------------------------------------------------- benchmark: 2 tests ---------------------------------------------------------------------------------
Name (time in s)             Min                Max               Mean            StdDev             Median               IQR            Outliers     OPS            Rounds  Iterations
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
foo3[50]       1.3493 (1.0)       1.3493 (1.0)       1.3493 (1.0)      0.0000 (1.0)       1.3493 (1.0)      0.0000 (1.0)           0;0  0.7411 (1.0)           1           1
foo4[500]     10.6643 (7.90)     10.6643 (7.90)     10.6643 (7.90)     0.0000 (1.0)      10.6643 (7.90)     0.0000 (1.0)           0;0  0.0938 (0.13)          1           1
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

no-logs-no-trace:
-------------------------------------------------------------------------------- benchmark: 2 tests -------------------------------------------------------------------------------
Name (time in s)            Min               Max              Mean            StdDev            Median               IQR            Outliers     OPS            Rounds  Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
foo5[50]      1.0546 (1.0)      1.0546 (1.0)      1.0546 (1.0)      0.0000 (1.0)      1.0546 (1.0)      0.0000 (1.0)           0;0  0.9483 (1.0)           1           1
foo6[500]     8.0793 (7.66)     8.0793 (7.66)     8.0793 (7.66)     0.0000 (1.0)      8.0793 (7.66)     0.0000 (1.0)           0;0  0.1238 (0.13)          1           1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

logs-info:
---------------------------------------------------------------------------------- benchmark: 2 tests ---------------------------------------------------------------------------------
Name (time in s)             Min                Max               Mean            StdDev             Median               IQR            Outliers     OPS            Rounds  Iterations
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
foo7[50]       1.3804 (1.0)       1.3804 (1.0)       1.3804 (1.0)      0.0000 (1.0)       1.3804 (1.0)      0.0000 (1.0)           0;0  0.7244 (1.0)           1           1
foo8[500]     11.1005 (8.04)     11.1005 (8.04)     11.1005 (8.04)     0.0000 (1.0)      11.1005 (8.04)     0.0000 (1.0)           0;0  0.0901 (0.12)          1           1
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

logs-info-no-trace:
-------------------------------------------------------------------------------- benchmark: 2 tests -------------------------------------------------------------------------------
Name (time in s)            Min               Max              Mean            StdDev            Median               IQR            Outliers     OPS            Rounds  Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
foo9[50]      1.2701 (1.0)      1.2701 (1.0)      1.2701 (1.0)      0.0000 (1.0)      1.2701 (1.0)      0.0000 (1.0)           0;0  0.7874 (1.0)           1           1
foo10[500]     8.1692 (6.43)     8.1692 (6.43)     8.1692 (6.43)     0.0000 (1.0)      8.1692 (6.43)     0.0000 (1.0)           0;0  0.1224 (0.16)          1           1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

^^ There all those numbers in parenthesis reflect the relative point in time or duration it took to calculate the respective stat? Note I know that in this particular case std dev, mean, max are pretty much irrelevant since only 1 round of 1 iteration is being performed. But I'm just trying to make sure I understand what you're saying correctly.

[ionelmc]

foo1[50]       2.0069 (1.0)
foo2[500]     15.4276 (7.69) 

That reads as: foo2[500] is 7.69 times more slower than foo1[50] regarding minimum durations. Clear as mud? :D

[hamin]

@ionelmc ah I get it! That's actually very useful and yes perhaps 1x might make more sense. Also I think you still might want to explain that somewhere in the docs or readme. Awesome! Thank you so much for the explanation. I'll go ahead and close the issue :)

[ionelmc]

So I don't forget to add some extra docs.