FrameworkBenchmarks

FrameworkBenchmarks copied to clipboard

In https://github.com/TechEmpower/FrameworkBenchmarks/pull/6966 I noticed a comment above the /plaintext route is:

// For performance purposes, this will successfully match '/p' to '/plaintext' and '/pickle'. Use with caution

We're requiring that the requests be made to /plaintext but we're not verifying in the toolset whether /p -> /plaintex returns Hello, World!. I'd like to add this verification to the new toolset @msmith-techempower

Will be good to verify the process priority too. ps al

Will be good to verify the process priority too. ps al

Is there a problem with running at a higher priority? This is nice for scheduling optimization. I think it is normal to elevate the priority of one process over another and do it all the time on my systems to ensure more consistent performance of my applications.

As for the original issue, In hindsight, that routing short-circuit was a mistake. While I want my framework's performance to improve, I would also like to play on an even playing field and am not interested in tiny, fractional gains like that anymore. I'm working hard on these improvements.

The problem with running at a higher priority, that personally I will not do it.

Is that all the frameworks will use it or none at all.

Priority mostly helps with max latency, not throughput. On dedicated servers I estimate it will contribute less than 2% to throughput. I care about latency almost as much as throughput though so I attempt to set it. No one really needs to set it unless they want more consistency across requests.

IMHO all OS-level optimizations should be fair game, be it process priorities, processor affinities, usage of io_uring, etc. (and no, personally I would not use all of them). Probably the only line that should be drawn is if something requires special privileges - for example, setting a real-time scheduling policy such as SCHED_FIFO for a process would be excluded. Incidentally, if you check @talawahtech's excellent blog post (Thanks a lot, mate!) in #6597, most of the significant optimizations are in fact privileged.

If a framework is not using a particular OS-level optimization, then its developers should think long and hard about that (and possibly remedy it). I think that is pretty much one of the motivations for the existence of the benchmarks in this repository.

As an aside - any usage of sudo and setting capabilities should be regarded with suspicion (And this is not an exhaustive list!). As for what particular OS-level optimizations a framework is using, it should be straightforward to discover that with a strace session.