FrameworkBenchmarks icon indicating copy to clipboard operation
FrameworkBenchmarks copied to clipboard

No more fakes

Open joanhey opened this issue 10 months ago • 10 comments

Please, no more http: length hard encoded.

No more template response cache !!! The template is OK to cache, but the response for the fortunes is NOT OK.

https://github.com/TechEmpower/FrameworkBenchmarks/pull/9560#issuecomment-2607279905

joanhey avatar Jan 31 '25 16:01 joanhey

mrhttp also non realistic implementation: https://github.com/TechEmpower/FrameworkBenchmarks/issues/9055

remittor avatar Feb 12 '25 13:02 remittor

Now the devs of the framework decide if it's realistic or not.

We need to implement more tests before start the benchmark !!

https://github.com/TechEmpower/FrameworkBenchmarks/issues/9440#issuecomment-2518472173

joanhey avatar Feb 12 '25 23:02 joanhey

The template is OK to cache, but the response for the fortunes is NOT OK.

I wonder if we should make fortunes response dynamic to prevent this. Maybe store more fortunes in the database and pick random fortunes, similar to the query test?

p8 avatar Feb 13 '25 15:02 p8

Validate response status codes #9605

joanhey avatar Feb 26 '25 17:02 joanhey

Shouldn't pre-computed HTTP headers trigger "Stripped" classification? #8205

joanhey avatar Feb 26 '25 18:02 joanhey

@p8 https://github.com/TechEmpower/FrameworkBenchmarks/issues/9578#issuecomment-2656955552 https://github.com/TechEmpower/FrameworkBenchmarks/issues/6529 https://github.com/TechEmpower/FrameworkBenchmarks/issues/6788 https://github.com/TechEmpower/FrameworkBenchmarks/issues/6883

joanhey avatar Feb 26 '25 18:02 joanhey

The aspnetcore framework precomputes the header length as well:

https://github.com/TechEmpower/FrameworkBenchmarks/blob/c82a834c5e8aeae14c8203cc05beaaa18dcc7cfc/frameworks/CSharp/aspnetcore-mono/PlatformBenchmarks/BenchmarkApplication.Json.cs#L14-L18

https://github.com/TechEmpower/FrameworkBenchmarks/blob/c82a834c5e8aeae14c8203cc05beaaa18dcc7cfc/frameworks/CSharp/aspnetcore-mono/PlatformBenchmarks/BenchmarkApplication.Json.cs#L12

https://github.com/TechEmpower/FrameworkBenchmarks/blob/c82a834c5e8aeae14c8203cc05beaaa18dcc7cfc/frameworks/CSharp/aspnetcore/src/Platform/BenchmarkApplication.Plaintext.cs#L13-L16

p8 avatar Mar 24 '25 21:03 p8

aspnetcore

image

Image

https://github.com/TechEmpower/FrameworkBenchmarks/blob/c82a834c5e8aeae14c8203cc05beaaa18dcc7cfc/frameworks/CSharp/aspnetcore/src/Platform/BenchmarkApplication.Json.cs#L19-L23

The benchmark uses a custom implementation of the Date header generation:

https://github.com/TechEmpower/FrameworkBenchmarks/blob/c82a834c5e8aeae14c8203cc05beaaa18dcc7cfc/frameworks/CSharp/aspnetcore/src/Platform/BenchmarkApplication.Json.cs#L25-L30

But in fact, we should test the implementation that comes with the original framework:

https://github.com/dotnet/aspnetcore/blob/main/src/Servers/Kestrel/Core/src/Internal/Http/DateHeaderValueManager.cs

https://github.com/dotnet/aspnetcore/blob/8645583a97fa8140da0a588bbd00c5e9cf13983c/src/Servers/Kestrel/Core/src/Internal/Http/HttpProtocol.cs#L1263-L1269

remittor avatar Mar 25 '25 05:03 remittor

Please, no more http: length hard encoded.

No more template response cache !!! The template is OK to cache, but the response for the fortunes is NOT OK.

#9560 (comment)

It will be really nice

Unfortunately, it will be a massive reduction in a number of approved frameworks. A lot of solutions just use dirty hacks to show higher values.

People sometimes are protecting this position with "Nginx-like proxy will be a protection layer and our server with logic can do whatever we want and we don't care about proper HTTP implementation"

cyrusmsk avatar Aug 10 '25 14:08 cyrusmsk

If you need a Nginx proxy to present a proper HTTP implementation, then the Nginx proxy needs to be in the test (welcome to the bottom of the charts, lol).

Dmole avatar Sep 18 '25 01:09 Dmole