[Question] How are ranks (in bar chart) computed

[waghanza]

Hi,

When we navigate on https://www.techempower.com/benchmarks/, the default page the last roud result on bar chart.

For example, we have 100% for libreactor and 3.8% for spark on https://www.techempower.com/benchmarks/#section=data-r18&hw=cl&test=plaintext

For the details (Data Table), we have

	256 (concurrency)	1024 (concurrency)	4096 (concurrency)	16384 (concurrency)
libreactor	1867993	1853616	1652369	1307490
spark	70972	71035	70276	67051

My question is : how can the above results show that we have 3.8% for spark, if libreactor is the maximum (100%) ?

Regards,

[msmith-techempower]

71,035 / 1,867,993 = ‭0.0380274444283249‬

iirc, we use the highest value among all concurrencies to calculate.

EDIT: Basically, if libreactor is the top performer, then everything is operating at a percentage of its performance. When you look at the best RPS (requests per second) for that test for spark, it is performing at 3.8% of the practical maximum, libreactor.

[waghanza]

then, the calculation you make is mixing concurrency level ?

I mean :

• 71 035 is on concurrency 1024 for spark
• 1 867 993 is on concurrency 256 for libreactor

However, if we consider all level of latency, spark is :

• ~3.80% for concurrency 256
• ~3.83% for concurrency 1024
• ~4.25% for concurrency 4096
• ~5.13% for concurrency 16384

which leads to an average of 4.25% for spark.

Mixing values for different concurrency levels could the lead to some results. So I'm afraid, this method could lead to mis-understandable results.

btw, thank you for you clarification about the 3.8%. I think this information could be documented.

Regards,

[NateBrady23]

@bhauer Maybe it's worth pointing out how we calculate these percentages and the usage of "best across concurrency levels" in https://www.techempower.com/benchmarks/#section=motivation under Results?

[bhauer]

Maybe it's worth pointing out how we calculate these percentages and the usage of "best across concurrency levels" in https://www.techempower.com/benchmarks/#section=motivation under Results?

That would probably help clarify things. It is indeed using the peak performance for each framework from the set of concurrency levels.