FrameworkBenchmarks

FrameworkBenchmarks copied to clipboard

Unstable Performance Among Some Java Test Implementations

10

I was troubleshooting what I believed to be a performance degradation in `Gemini` (and spent a lot of time doing so) when I believe I came to the realization that...

msmith-techempower

New benchmark type porposal - JSON deserialization over POST

1

These benchmark tests have been my go to place for any kind of performance comparison. Really love the thoroughness here. Lately, I have come across cases where I want to...

grssam

[Question] How are ranks (in bar chart) computed

4

Hi, When we navigate on https://www.techempower.com/benchmarks/, the default page the last roud result on bar chart. For example, we have **100%** for `libreactor` and **3.8%** for `spark` on https://www.techempower.com/benchmarks/#section=data-r18&hw=cl&test=plaintext For...

waghanza

Enhancement of the verification procedure

16

The objective is to identify possible improvements of the [new verification procedure](https://github.com/TechEmpower/FrameworkBenchmarks/pull/5145): ### Too strict checking for db queries? Sometimes the test fails for 1 or 2 missing db queries...

jcheron

Is it correct to hard-code the Content-Length ?

6

Looking code from others frameworks, I found that: https://github.com/TechEmpower/FrameworkBenchmarks/blob/master/frameworks/C%2B%2B/ulib/src/fortune.h#L151-L182 https://github.com/TechEmpower/FrameworkBenchmarks/blob/master/frameworks/C%2B%2B/ulib/src/json.usp#L25-L36 Perhaps others frameworks do it too. In the Requirements I don't see anything, only that: The response content length should...

joanhey

Plan visualization for resource statistics gathered by dstat

9

We use `dstat` to capture application server resource utilization statistics while we are measuring throughput and latency via the load generator. The raw data from `dstat` is stored at the...

bhauer

We are considering adding a tag to framework metadata to indicate that the maintainers of a given framework have **verified** the implementation. We believe this would be useful to readers...

bhauer

Let us drill down by concurrency level in latency

1

I wonder why is there no concurrency level column/filter?  I think it's not just me who would like to see latency over lower concurrency,...

ostrolucky

Tests that need updating before next round

11

This is a living list of tests that need to be updated or removed before we record our next official round results. I'll do my best to ping the most...

NateBrady23

As noted in #5079 some of the Python tests are going about the Updates test implementation incorrectly. ```python async def database_updates( req: MultipleDatabaseRequest ) -> MultipleDatabaseResponse: num_queries = get_num_queries(req.query['queries']) updates...

NateBrady23