FrameworkBenchmarks icon indicating copy to clipboard operation
FrameworkBenchmarks copied to clipboard

Perl Kelp: Fix and modernize

Open bbrtj opened this issue 1 year ago • 3 comments

I am the current maintainer of the Perl Kelp framework. This PR fixes and rewrites its benchmark code. I took the approach to only fix what has already been done by the author, so I neither added the new cache test or more databases (like Postgre) nor removed stuff that were already there (like t/main.t test).

Status:

  • kelp (MariaDB) passes tests with tfb
  • kelp-mongodb passes tests with tfb
  • bundled test file passes with prove -l

I haven't contributed any code here up until now, so let me know if anything else needs to be changed.

bbrtj avatar Jun 13 '24 07:06 bbrtj

I've chosen that the server should replace processes by new ones after handling 10000 requests. While it would surely perform better with an even higher number, I think this is a sensible production value. On top of that, it's the same as the default number of requests handled by Hypnotoad (mojolicious), so it will be more accurate overview of the performance differences between the two.

Perl frameworks here use very differing values of that config. dancer and web-simple benchmarks use Starman server with a default of 1000 requests before process shutdown, Plack benchmark uses Gazelle (same as Kelp here) with 10000000.

All Perl benchmarks should use the same value (possibly higher than ten thousand to better compete against other languages). This is critical when it comes to measuring the performance - forking new processes not only takes time but also makes the program spend time recalculating all the cache, reconnecting to databases etc. It will be hard for a program to achieve high requests per second if it has to spend significant time on internal process bookkeeping.

Edit: correction, Mojolicious currently uses infinity requests per process.

bbrtj avatar Jun 13 '24 10:06 bbrtj

  • Added tests on multiple Perl webservers, 8 configurations in total now
  • removed the .toml file, as it seems to not be required anymore (tell me if I'm wrong)
  • increased the number of requests per process to 100,000

bbrtj avatar Jun 13 '24 17:06 bbrtj

I'm finished, ready for review and merge. After it's merged, I'll make another PR which will increase requests per process of all perl framework benchmarks to the same value (one hundred thousand).

bbrtj avatar Jun 14 '24 07:06 bbrtj