swoole-src icon indicating copy to clipboard operation
swoole-src copied to clipboard

A bug occurred in Swoole-v5.1.2, please report it

Open error1001es opened this issue 9 months ago • 26 comments

Please answer these questions before submitting your issue.

  1. What did you do? If possible, provide a simple script for reproducing the error.

update swoole in docker php ext

  1. What did you expect to see?

all working

  1. What did you see instead?

doesnt response laravel octane with swoole

  1. What version of Swoole are you using (show your php --ri swoole)?

it says 5.1.2

  1. What is your machine environment used (show your uname -a & php -v & gcc -v) ? Linux 1baab60a7013 5.15.146.1-microsoft-standard-WSL2 #1 SMP Thu Jan 11 04:09:03 UTC 2024 x86_64 Linux PHP 8.3.7 (cli) (built: May 10 2024 21:59:17) (NTS) Copyright (c) The PHP Group Zend Engine v4.3.7, Copyright (c) Zend Technologies /app $ gcc -v /bin/sh: gcc: not found

[2024-05-12 09:13:10 #19.0] WARNING Server::check_worker_exit_status(): worker(pid=30, id=2) abnormal exit, status=0, signal=11 A bug occurred in Swoole-v5.1.2, please report it. The Swoole developers probably don't know about it, and unless you report it, chances are it won't be fixed. You can read How to report a bug doc before submitting any bug reports:

https://github.com/swoole/swoole-src/blob/master/.github/ISSUE.md Please do not send bug reports in the mailing list or personal letters. The issue page is also suitable to submit feature requests.

OS: Linux 5.15.146.1-microsoft-standard-WSL2 #1 SMP Thu Jan 11 04:09:03 UTC 2024 x86_64 GCC_VERSION: 13.2.1 20231014 OPENSSL_VERSION: OpenSSL 3.1.4 24 Oct 2023 PHP_VERSION : 8.3.7

error1001es avatar May 12 '24 09:05 error1001es

RUN install-php-extensions pcntl sockets swoole pdo_mysql

this command i installed swoole

error1001es avatar May 12 '24 09:05 error1001es

Do you have the xdebug extension? If so, try not using it and see.

NathanFreeman avatar May 13 '24 01:05 NathanFreeman

no, i dont use xdebug

error1001es avatar May 13 '24 11:05 error1001es

it's error only in 5.1.2, 5.1.1 works fine

error1001es avatar May 13 '24 11:05 error1001es

Same here, using Laravel, Octane, Swoole

Image built using the following Dockerfile:

FROM php:8.3-cli-alpine

WORKDIR /var/www/html

RUN curl -sSLf \
    -o /usr/local/bin/install-php-extensions \
    https://github.com/mlocati/docker-php-extension-installer/releases/download/2.2.11/install-php-extensions \
    && echo "4d8a1bbd2d53a4ce8d194647da9dc2ff37768a26cdb46f459bfc7642d170eb85147534e850bcb503b7b38f04ddb6228f2ef0e50619349ba02fe79aa138452cbe  /usr/local/bin/install-php-extensions" | sha512sum -c - \
    && chmod +x /usr/local/bin/install-php-extensions \
    && install-php-extensions \
    zip \
    pdo_mysql \
    pgsql \
    pdo_pgsql \
    redis \
    calendar \
    gd \
    exif \
    swoole \
    pcntl \
    && apk add --no-cache --virtual .build-deps npm \
    && npm install -g svgo \
    && apk add --no-cache \
    su-exec \
    supervisor \
    jpegoptim \ 
    optipng \
    pngquant \
    gifsicle \
    ffmpeg \
    && apk del .build-deps

Trying to access the project with Octane & Swoole:

[2024-05-13 14:10:43 #31274.0]  WARNING Server::check_worker_exit_status(): worker(pid=31408, id=16) abnormal exit, status=0, signal=11
A bug occurred in Swoole-v5.1.2, please report it.
The Swoole developers probably don't know about it,
and unless you report it, chances are it won't be fixed.
You can read How to report a bug doc before submitting any bug reports:
>> https://github.com/swoole/swoole-src/blob/master/.github/ISSUE.md 
Please do not send bug reports in the mailing list or personal letters.
The issue page is also suitable to submit feature requests.

OS: Linux 6.1.0-20-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.85-1 (2024-04-11) x86_64
GCC_VERSION: 13.2.1 20231014
OPENSSL_VERSION: OpenSSL 3.1.4 24 Oct 2023
PHP_VERSION : 8.3.7

Additional info:

/var/www/html # php --ri swoole

swoole

Swoole => enabled
Author => Swoole Team <[email protected]>
Version => 5.1.2
Built => May 13 2024 13:08:15
coroutine => enabled with boost asm context
epoll => enabled
eventfd => enabled
signalfd => enabled
spinlock => enabled
rwlock => enabled
openssl => OpenSSL 3.1.4 24 Oct 2023
dtls => enabled
http2 => enabled
json => enabled
c-ares => 1.27.0
zlib => 1.3.1
brotli => E16781312/D16781312
mutex_timedlock => enabled
pthread_barrier => enabled
futex => enabled
mysqlnd => enabled
async_redis => enabled

Directive => Local Value => Master Value
swoole.enable_coroutine => On => On
swoole.enable_library => On => On
swoole.enable_fiber_mock => Off => Off
swoole.enable_preemptive_scheduler => Off => Off
swoole.display_errors => On => On
swoole.use_shortname => On => On
swoole.unixsock_buffer_size => 8388608 => 8388608
/var/www/html # uname -a
Linux 6883120929f9 6.1.0-20-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.85-1 (2024-04-11) x86_64 Linux
/var/www/html # php -v
PHP 8.3.7 (cli) (built: May 10 2024 21:59:17) (NTS)
Copyright (c) The PHP Group
Zend Engine v4.3.7, Copyright (c) Zend Technologies
/var/www/html # gcc -v
/bin/ash: gcc: not found

Can confirm it's working on v5.1.1 (previous build)

modrink avatar May 13 '24 14:05 modrink

@modrink I can not reproduce it, Is this a consistently reproducible scenario? I am directly accessing 127.0.0.1:8000 without any business logic.

NathanFreeman avatar May 15 '24 02:05 NathanFreeman

We have Cloudflare & Traefik V3 in front, but I'll test more when I get the chance. I was not able to reproduce this locally (only on server with the proxy). But Swoole v5.1.1 works and OpenSwoole latest works as well.

With Traefik the occurring errors are HTTP 499 and 502

modrink avatar May 15 '24 14:05 modrink

Hi all,

I'm afraid we have not yet had the time to fully make a reproducible case without sharing sensitive details, but I can confirm the issue is present in Swoole 5.1.2 and that reverting back to 5.1.1 solves the issue.

We have tested setting up a clean Laravel project with Octane using Swoole 5.1.2 inside a docker image running from php:8.2-cli-alpine and using mlocati/php-extension-installer to install Swoole. When reaching Swoole on port 8000 the service keeps running, the same goes for when we access it through NGINX inside the image as well.

The NGINX config inside the image looks like this:

map $http_upgrade $connection_upgrade {
    default upgrade;
    ''      close;
}
 
server {
    listen 80;
    listen [::]:80;
    server_name domain.com;
    server_tokens off;
    root /home/forge/domain.com/public;
 
    index index.php;
 
    charset utf-8;
 
    location /index.php {
        try_files /not_exists @octane;
    }
 
    location / {
        try_files $uri $uri/ @octane;
    }
 
    location = /favicon.ico { access_log off; log_not_found off; }
    location = /robots.txt  { access_log off; log_not_found off; }
 
    access_log off;
    error_log  /var/log/nginx/domain.com-error.log error;
 
    error_page 404 /index.php;
 
    location @octane {
        set $suffix "";
 
        if ($uri = /index.php) {
            set $suffix ?$query_string;
        }
 
        proxy_http_version 1.1;
        proxy_set_header Host $http_host;
        proxy_set_header Scheme $scheme;
        proxy_set_header SERVER_PORT $server_port;
        proxy_set_header REMOTE_ADDR $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection $connection_upgrade;
 
        proxy_pass http://127.0.0.1:8000$suffix;
    }
}

The problem actually occurs when we are running the requests through the Kubernetes Ingress Controller which is also using NGINX behind the scenes. After the first request is made through the Kubernetes Ingress Controller the Swoole service shuts down, after this we are also unable to reach the Swoole endpoint directly within the environment as well.

I can concur with @modrink that the issue is that the client closes the connection (HTTP 499), so there seems to be something wrong with the implementation in the HTTP server that was introduced in 5.1.2 and that it's triggered mostly when using HTTP Proxies like NGNIX (with certain settings), Traefik and Cloudflare.

As mentioned we have not yet been able to reproduce this in a way to share yet.

robinnydahl avatar May 16 '24 06:05 robinnydahl

@robinnydahl @modrink Perhaps some commits led to this issue.I'll take some time to test it.

NathanFreeman avatar May 16 '24 08:05 NathanFreeman

This happens to me as well. swoole 5.1.2 + xdebug 3.3.2

sandbox-sb.34i5g-swoole8.1 | [2024-05-17 13:07:00 #1.0] WARNING Server::check_worker_exit_status(): worker(pid=154, id=9) abnormal exit, status=0, signal=11 sandbox-sb.34i5g-swoole8.1 | A bug occurred in Swoole-v5.1.2, please report it. sandbox-sb.34i5g-swoole8.1 | The Swoole developers probably don't know about it, sandbox-sb.34i5g-swoole8.1 | and unless you report it, chances are it won't be fixed. sandbox-sb.34i5g-swoole8.1 | You can read How to report a bug doc before submitting any bug reports: sandbox-sb.34i5g-swoole8.1 | >> https://github.com/swoole/swoole-src/blob/master/.github/ISSUE.md sandbox-sb.34i5g-swoole8.1 | Please do not send bug reports in the mailing list or personal letters. sandbox-sb.34i5g-swoole8.1 | The issue page is also suitable to submit feature requests. sandbox-sb.34i5g-swoole8.1 | sandbox-sb.34i5g-swoole8.1 | OS: Linux 6.8.8-300.fc40.x86_64 #1 SMP PREEMPT_DYNAMIC Sat Apr 27 17:53:31 UTC 2024 x86_64 sandbox-sb.34i5g-swoole8.1 | GCC_VERSION: 11.4.0 sandbox-sb.34i5g-swoole8.1 | OPENSSL_VERSION: OpenSSL 3.0.2 15 Mar 2022 sandbox-sb.34i5g-swoole8.1 | PHP_VERSION : 8.1.2-1ubuntu2.17 sandbox-sb.34i5g-swoole8.1 |

linuxd3v avatar May 17 '24 17:05 linuxd3v

@linuxd3v Can you provide a simple script for reproducing the error?

NathanFreeman avatar May 19 '24 07:05 NathanFreeman

@linuxd3v I found that the max_nesting_level parameter of xdebug is set to 512. If a certain loop exceeds 512 times, a PHP Fatal error will be thrown, terminating the process.

NathanFreeman avatar May 19 '24 09:05 NathanFreeman

@NathanFreeman I've tried setting max_nesting_level to 50000 - but still seeing the same issue.
Actually - seeing the same issue with Swoole-v5.1.0 and Swoole-v5.1.1 Also tried xdebug 3.3.1 - but with the same result as xdebug 3.3.2.

linuxd3v avatar May 20 '24 13:05 linuxd3v

@linuxd3v I don't have a good solution for xdebug. There are too many configurations for xdebug, unless you can find out through the logs which specific configuration is causing PHP to throw a fatal error.

NathanFreeman avatar May 20 '24 13:05 NathanFreeman

@error1001es @robinnydahl @modrink I can not reproduce it. Please use valgrind or gdb to trace the program execution. https://wiki.swoole.com/en/#/other/issue?id=about-segmentation-fault-core-dump https://wiki.swoole.com/en/#/other/issue?id=memory-analysis-highly-recommended

NathanFreeman avatar May 24 '24 12:05 NathanFreeman

@robinnydahl described the exact problem that we are facing with Swoole v5.1.2. I took some time to get the data you requested.

Call stack:

#0  0x0000565154971d20 in zend_hash_real_init_mixed ()
#1  0x0000565154973a35 in zend_hash_str_update ()
#2  0x00007f569d472163 in ?? () from /usr/local/lib/php/extensions/no-debug-non-zts-20230831/swoole.so
#3  0x00007f569d52fb70 in swoole_http_parser_execute () from /usr/local/lib/php/extensions/no-debug-non-zts-20230831/swoole.so
#4  0x00007f569d3dc4ec in php_swoole_http_server_onReceive(swoole::Server*, swoole::RecvData*) () from /usr/local/lib/php/extensions/no-debug-non-zts-20230831/swoole.so
#5  0x00007f569d4aecd7 in swoole::Server::worker_accept_event(swoole::DataHead*) () from /usr/local/lib/php/extensions/no-debug-non-zts-20230831/swoole.so
#6  0x00007f569d4af8a5 in ?? () from /usr/local/lib/php/extensions/no-debug-non-zts-20230831/swoole.so
#7  0x00007f569d4eda91 in swoole::ReactorEpoll::wait(timeval*) () from /usr/local/lib/php/extensions/no-debug-non-zts-20230831/swoole.so
#8  0x00007f569d4afe81 in swoole::Server::start_event_worker(swoole::Worker*) () from /usr/local/lib/php/extensions/no-debug-non-zts-20230831/swoole.so
#9  0x00007f569d4b3e9b in swoole::Server::spawn_event_worker(swoole::Worker*) () from /usr/local/lib/php/extensions/no-debug-non-zts-20230831/swoole.so
#10 0x00007f569d4b6082 in ?? () from /usr/local/lib/php/extensions/no-debug-non-zts-20230831/swoole.so
#11 0x00007f569d50ffef in swoole_fork_exec(std::function<void ()> const&) () from /usr/local/lib/php/extensions/no-debug-non-zts-20230831/swoole.so
#12 0x00007f569d4b661c in swoole::Server::start_manager_process() () from /usr/local/lib/php/extensions/no-debug-non-zts-20230831/swoole.so
#13 0x00007f569d4cd3f9 in swoole::ProcessFactory::start() () from /usr/local/lib/php/extensions/no-debug-non-zts-20230831/swoole.so
#14 0x00007f569d4c0c8a in swoole::Server::start() () from /usr/local/lib/php/extensions/no-debug-non-zts-20230831/swoole.so
#15 0x00007f569d47f7e6 in ?? () from /usr/local/lib/php/extensions/no-debug-non-zts-20230831/swoole.so
#16 0x00005651549d576d in execute_ex ()
#17 0x00005651549d7883 in zend_execute ()
#18 0x00005651549645a8 in zend_execute_scripts ()
#19 0x00005651548f9e80 in php_execute_script ()
#20 0x0000565154a4d245 in ?? ()
#21 0x0000565154647604 in ?? ()
#22 0x00007f569ebbd70a in libc_start_main_stage2 (main=0x565154647310, argc=3, argv=0x7fffd594ebc8) at src/env/__libc_start_main.c:95
#23 0x000056515464769f in _start ()

It's worth noting that the trace doesn't always include frame 0. Sometimes it stops at zend_hash_str_update () (What is frame 1 in the trace above).

It's also worth noting that instead of PHP 8.2, we're using the php:8.3.7-cli-alpine20 base image. We also run Swoole using Laravel Octane.

Our Dockerfile is based on this example Dockerfile: https://github.com/swoole/docker-swoole/blob/master/dockerfiles/5.1.2/php8.3/alpine/Dockerfile

lv2u avatar May 30 '24 15:05 lv2u

Please answer these questions before submitting your issue.

  1. What did you do? If possible, provide a simple script for reproducing the error.

update swoole in docker php ext

  1. What did you expect to see?

all working

  1. What did you see instead?

doesnt response laravel octane with swoole

  1. What version of Swoole are you using (show your php --ri swoole)?

it says 5.1.2

  1. What is your machine environment used (show your uname -a & php -v & gcc -v) ? Linux 1baab60a7013 5.15.146.1-microsoft-standard-WSL2 编译出错 #1 SMP Thu Jan 11 04:09:03 UTC 2024 x86_64 Linux PHP 8.3.7 (cli) (built: May 10 2024 21:59:17) (NTS) Copyright (c) The PHP Group Zend Engine v4.3.7, Copyright (c) Zend Technologies /app $ gcc -v /bin/sh: gcc: not found

[2024-05-12 09:13:10 #19.0] WARNING Server::check_worker_exit_status(): worker(pid=30, id=2) abnormal exit, status=0, signal=11 A bug occurred in Swoole-v5.1.2, please report it. The Swoole developers probably don't know about it, and unless you report it, chances are it won't be fixed. You can read How to report a bug doc before submitting any bug reports:

https://github.com/swoole/swoole-src/blob/master/.github/ISSUE.md Please do not send bug reports in the mailing list or personal letters. The issue page is also suitable to submit feature requests.

OS: Linux 5.15.146.1-microsoft-standard-WSL2 #1 SMP Thu Jan 11 04:09:03 UTC 2024 x86_64 GCC_VERSION: 13.2.1 20231014 OPENSSL_VERSION: OpenSSL 3.1.4 24 Oct 2023 PHP_VERSION : 8.3.7

Same problem here.

OS: Ubuntu PHP Version: 8.1.28 Swoole Version: 5.1.2 Binary generator CLI: I'm use Static PHP CLI

Context description: I'm using the hyperf application binary to launch my server application, but the worker die when receiving a request from the browser. Apparently when I use an HTTP client it works normally

Command start: bin/app start image

After make a request in browser (The request seems to go into a loop and the logs show me this error message) image

The same request being done in HTTP Client (or curl) work fine image

HallexCosta avatar Jun 01 '24 06:06 HallexCosta

@HallexCosta @lv2u @error1001es @robinnydahl @modrink We have released Swoole v5.1.3, optimized a dynamic array. You can try Swoole v5.1.3 to see if this issue still exists.

NathanFreeman avatar Jun 06 '24 07:06 NathanFreeman

@NathanFreeman confirmed the issue has been resolved with v5.1.3 btw the Swoole was serving the request fine directly but failed behind an LB or Nginx

salkhwlani avatar Jun 07 '24 08:06 salkhwlani

What error messages did you get? @salkhwlani

NathanFreeman avatar Jun 07 '24 08:06 NathanFreeman

@NathanFreeman worker[0] error: exitCode=0, signal=11

salkhwlani avatar Jun 07 '24 08:06 salkhwlani

Are you using xdebug, or can you check the stack output using valgrind? @salkhwlani

NathanFreeman avatar Jun 07 '24 08:06 NathanFreeman

No, I don't use Xdebug; the issue is resolved once I update the swoole from 5.1.2 to version v5.1.3

salkhwlani avatar Jun 07 '24 08:06 salkhwlani

Confirming Swoole v5.1.3 fixes the issues described in this report.

lv2u avatar Jun 10 '24 09:06 lv2u

confirming swoole-5.1.3 fixes it!

error1001es avatar Jun 10 '24 13:06 error1001es

Sorry for taking so long to answer. Confirming swoole-5.1.3 fixes it!

HallexCosta avatar Jun 16 '24 05:06 HallexCosta