Request for reproducible benchmarks

[oraluben]

• Node Version: v16.15.1
• Midway Version(Decorator/Core):
• Component Name/Version:
• Platform:
• Mini Showcase Repository:

We're evaluating the performance of node frameworks, but the benchmark(s) of midway seems to lack document (there's benchmark/ and scripts/benchmark.sh) and bounded to specific node version (only run benchmark CI with node 14).

While the benchmark/ seems updated, I cannot get a valuable result from it. It runs without error but returns a zero qps. The scripts/benchmark.sh is used by Github Actions CI but I cannot run it with node 16. I made some dirty updates trying to get it working, but not all issues are solved: #2228.

It would be nice to have a stable benchmark with step-by-step instruction.

[czy88840616]

We set this action to avoid memory leak and will be fix it soon.

[oraluben]

While the benchmark/ seems updated, I cannot get a valuable result from it. It runs without error but returns a zero qps.

Actually, I can get a positive QPS on some of my dev envs, and zero on other dev envs and my local env. Do you have any idea about this? I also pushed some updates about benchmark/readme.md that you may find useful.

[czy88840616]

While the benchmark/ seems updated, I cannot get a valuable result from it. It runs without error but returns a zero qps.

Actually, I can get a positive QPS on some of my dev envs, and zero on other dev envs and my local env. Do you have any idea about this? I also pushed some updates about benchmark/readme.md that you may find useful.

maybe have some error when project start. You can got real qps after build and run with npm run start.

[oraluben]

ok it's still not working... the benchmark quit instantly after startup:

$ npm run start

> [email protected] start
> node bootstrap.js

2022-08-12 17:15:33,524 INFO 54924 [midway:bootstrap] current app started
2022-08-12 17:15:33,527 INFO 54924 [midway:bootstrap] exit with code:0
$ curl localhost:7001                       
curl: (7) Failed to connect to localhost port 7001 after 7 ms: Connection refused

Sometimes it seems to work because some stale server is listening on 7001 and returns result, it's not the benchmark server.

[czy88840616]

npm i & npm run build & npm run start

[oraluben]

thanks, it worked!

[github-actions[bot]]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.