nginx
Unexpectedly high cpu usage with tiny php-app
Bug Overview
We tried to switch a small service from php-fpm to nginx-unit, but this new setup consumes ~10 times more CPU than the previous one.
The app is very simple, it receives http request from client, then makes 3 requests to a Redis storage and send the response back to the client. The usual load is around 300 req/second.
- with php-fpm it consumes ~150mcpus (we use kubernetes)
- with nginx-unit it is 1000-2000 mcpus
Versions we tried: Most of the tests were made on the RPM from Remi's repo:
unit version: 1.34.2 configured as ./configure --libdir=/usr/lib64 --sbindir=/usr/sbin --prefix=/usr --statedir=/var/lib/unit --control=unix:/run/unit/control.sock --pid=/run/unit/unit.pid --runstatedir=/var/run --log=/var/log/unit/unit.log --logdir=/var/log --tmpdir=/var/tmp --user=unit --group=unit --openssl --cc-opt='-O2 -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -march=x86-64-v2 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection' --tests --njs --otel --modulesdir=/usr/lib64/unit/modules
But we alse tried 1.30 also from Remi, and official bookworm-based image 1.34.2-php8.2
The result is always the same.
Config we used:
{ "access_log": { "path": "/dev/null" }, "listeners": { "*:9000": { "pass": "routes" } }, "routes": [ { "match": { "uri": "/ping" }, "action": { "return": 200 } }, { "match": { "uri": "/announce.php" }, "action": { "pass": "applications/p2p/announce" } } ], "applications": { "p2p": { "type": "php", "stdout": "/dev/null", "stderr": "/proc/self/fd/2", "processes": { "max": 64, "spare": 64 }, "working_directory": "/var/www", "limits": { "timeout": 5, "requests": 5000 }, "targets": { "announce": { "root": "/var/www/public", "script": "announce.php" } } } } }
Expected Behavior
nginx-unit consumes the same amount of cpu time as php-fpm (or less)
Steps to Reproduce the Bug
Run application and check cpu consumption under the load of 300 req/sec
Environment Details
- Target deployment platform: bare-metal kubernetes cluster
- Target OS: worker nodes run Centos 7, image build on rockylinux 9, we also tried debian-bookworm based
- Version of this project or specific commit: 1.34.2 https://github.com/nginx/unit/commit/8ab74a8cc929272eb8683d3f6ab4cb406465fd34
- Version of any relevant project languages: Kubernetes v1.23.7 / PHP 8.4 (also tried 8.2)
Additional Context
No response
Hey, just out of curiosity,
- Do you see behaviour where the response is good/ cpu usage low when starting, but within a little time, it starts going up?
- Is it possible to test a simply php to echo something back without the redis coming into play, and see if you see the same behavour? I ask because almost exact use case and it turned out redis connections running out, causing delay, causing cpu usage to go high. Yours may be completely different, but just sharing my experience.
how many cores does your machine have?
unit is not cgroup aware so on kubernetes it will run with far too many threads and have very bad performance, see https://github.com/nginx/unit/issues/1042#issuecomment-2277740318
https://unit.nginx.org/configuration/#settings changing listen_threads to your cpu limit may improve the performance.
