Crashes with coroutines and curl 8.10.1
Since 1.10.2024, our application started to crash:
app-1 | [2024-10-04 06:52:16 $119.0] WARNING Server::check_worker_exit_status(): worker(pid=711, id=0) abnormal exit, status=0, signal=11
app-1 | A bug occurred in OpenSwoole-v22.1.2, please report it.
app-1 | Please submit bug report at:
app-1 | >> https://github.com/openswoole/swoole-src/issues
app-1 |
app-1 | OS: Linux 6.6.51-gentoo #1 SMP PREEMPT_DYNAMIC Fri Sep 13 23:06:02 UTC 2024 x86_64
app-1 | GCC_VERSION: 13.2.1 20240309
app-1 | OPENSSL_VERSION: OpenSSL 3.3.2 3 Sep 2024
app-1 | PHP_VERSION : 8.3.11
From our Dockerfile:
FROM php:8.3.11-alpine3.20
RUN apk upgrade --no-cache \
&& apk add --no-cache \
curl-dev
# Install extensions
RUN pecl install -D 'enable-openssl="yes" enable-hook-curl="yes"' openswoole-22.1.2
There is a lot of inter-service communication between the microservices via Guzzlehttp and it only happens with coroutines enabled.
That time, curl was upgraded in Alpine 3.20: https://gitlab.alpinelinux.org/alpine/aports/-/commit/e17ad295e0810e938f9c4d61c21e44eaf1a59b51 https://gitlab.alpinelinux.org/alpine/aports/-/commit/badfda8e31a0b6e87e78237d7ee4455b35cd4c10
After reverting back to Alpine 3.19 it works fine.
Alpine 3.19: curl 8.9.1 Alpine 3.20: curl 8.10.1
Maybe related: https://github.com/swoole/swoole-src/issues/5500 https://github.com/curl/curl/issues/15127
php --ri openswoole
openswoole
Open Swoole => enabled
Author => Open Swoole Group <[email protected]>
Version => 22.1.2
Built => Oct 4 2024 06:56:46
coroutine => enabled with boost asm context
epoll => enabled
eventfd => enabled
signalfd => enabled
spinlock => enabled
rwlock => enabled
openssl => OpenSSL 3.1.7 3 Sep 2024
dtls => enabled
hook-curl => enabled
zlib => 1.3.1
brotli => E16781312/D16781312
mutex_timedlock => enabled
pthread_barrier => enabled
Directive => Local Value => Master Value
openswoole.enable_coroutine => On => On
openswoole.enable_preemptive_scheduler => Off => Off
openswoole.display_errors => On => On
openswoole.unixsock_buffer_size => 8388608 => 8388608
uname -a
Linux 5ff868dd84d7 6.6.51-gentoo #1 SMP PREEMPT_DYNAMIC Fri Sep 13 23:06:02 UTC 2024 x86_64 Linux
php -v
PHP 8.3.11 (cli) (built: Sep 7 2024 01:13:27) (NTS)
Copyright (c) The PHP Group
Zend Engine v4.3.11, Copyright (c) Zend Technologies
with Zend OPcache v8.3.11, Copyright (c), by Zend Technologies
gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/libexec/gcc/x86_64-alpine-linux-musl/13.2.1/lto-wrapper
Target: x86_64-alpine-linux-musl
Configured with: /home/buildozer/aports/main/gcc/src/gcc-13-20231014/configure --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --build=x86_64-alpine-linux-musl --host=x86_64-alpine-l
inux-musl --target=x86_64-alpine-linux-musl --enable-checking=release --disable-cet --disable-fixed-point --disable-libstdcxx-pch --disable-multilib --disable-nls --disable-werror --disable-symvers
--enable-__cxa_atexit --enable-default-pie --enable-default-ssp --enable-languages=c,c++,d,objc,go,fortran,ada --enable-link-serialization=2 --enable-linker-build-id --disable-libssp --disable-libsa
nitizer --enable-shared --enable-threads --enable-tls --with-bugurl=https://gitlab.alpinelinux.org/alpine/aports/-/issues --with-system-zlib --with-linker-hash-style=gnu --with-pkgversion='Alpine 13
.2.1_git20231014'
Thread model: posix
Supported LTO compression algorithms: zlib
gcc version 13.2.1 20231014 (Alpine 13.2.1_git20231014)
Confirmed, same issue here. This is coredump stacktrace:
swoole::curl::Multi::create_socket (this=this@entry=0x7ffff1400950, cp=0x7ffff147f030, sockfd=sockfd@entry=10) at /tmp/swoole-src/ext-src/swoole_curl.cc:77
77 handle->socket = socket;
(gdb) bt
#0 swoole::curl::Multi::create_socket (this=this@entry=0x7ffff1400950, cp=0x7ffff147f030, sockfd=sockfd@entry=10) at /tmp/swoole-src/ext-src/swoole_curl.cc:77
#1 0x00007ffff5ed677a in swoole::curl::Multi::set_event (this=0x7ffff1400950, cp=0x7ffff147f030, socket_ptr=<optimized out>, sockfd=10, action=1) at /tmp/swoole-src/ext-src/swoole_curl.cc:103
#2 0x00007ffff5ed67b5 in swoole::curl::Multi::handle_socket (easy=<optimized out>, s=<optimized out>, action=<optimized out>, userp=<optimized out>, socketp=<optimized out>) at /tmp/swoole-src/ext-src/swoole_curl.cc:51
#3 0x00007ffff761c1f2 in ?? () from /usr/lib/libcurl.so.4
#4 0x00007ffff75ec4dd in ?? () from /usr/lib/libcurl.so.4
#5 0x00007ffff75ece74 in ?? () from /usr/lib/libcurl.so.4
#6 0x00007ffff75ecf95 in ?? () from /usr/lib/libcurl.so.4
#7 0x00007ffff75ed1bf in ?? () from /usr/lib/libcurl.so.4
#8 0x00007ffff761d0d3 in curl_multi_cleanup () from /usr/lib/libcurl.so.4
#9 0x00007ffff5fafd18 in swoole::curl::Multi::~Multi (this=0x7ffff1400950, __in_chrg=<optimized out>) at /tmp/swoole-src/ext-src/php_swoole_curl.h:112
#10 zif_swoole_native_curl_exec (execute_data=0x7ffff084a480, return_value=0x7ffff1400a00) at /tmp/swoole-src/thirdparty/php/curl/interface.cc:2591
#11 0x00005555559d19bf in execute_ex ()
#12 0x00007ffff5ecf73d in swoole::PHPCoroutine::main_func (arg=0x7fffffff9fa0) at /tmp/swoole-src/ext-src/swoole_coroutine.cc:772
#13 0x00007ffff5f384d4 in std::function<void (void*)>::operator()(void*) const (__args#0=<optimized out>, this=<optimized out>) at /usr/include/c++/13.2.1/bits/std_function.h:591
#14 swoole::coroutine::Context::context_func_v2 (transfer=...) at /tmp/swoole-src/src/coroutine/context.cc:163
#15 0x00005555558f78af in make_fcontext ()
#16 0x0000000000000000 in ?? ()
After migrating to swoole https://github.com/swoole/swoole-src/releases/tag/v5.1.5 it seems to be gone.
Fixed by upstream: php:8.3.11-alpine3.20 libcurl/8.9.1 libcurl/8.11.1