swoole-src icon indicating copy to clipboard operation
swoole-src copied to clipboard

Error WARNING ReactorEpoll::add()

Open jilcimar opened this issue 8 months ago • 18 comments

I encountered an issue when using a coroutine to send data to AWS SNS within my application. The error occurs at runtime with the following message:

WARNING	ReactorEpoll::add(): failed to add events[fd=26#0, type=27, events=512], Error: Bad file descriptor[9]

This happens when executing the coroutine using co(function () { ... })

Environment Details:

Image

jilcimar avatar Apr 03 '25 19:04 jilcimar

I have the same problem too

ElielFC avatar Apr 04 '25 19:04 ElielFC

Any update?

b-bianca avatar Apr 04 '25 19:04 b-bianca

Hi. Please tell me what framework you are using.
If possible, could you provide a reproducible code example? Thank you @jilcimar @ElielFC @b-bianca

NathanFreeman avatar Apr 05 '25 01:04 NathanFreeman

You need to provide a code example that can reproduce this issue. Additionally, if you are using extensions such as gRPC, there may be compatibility issues with Swoole.

matyhtf avatar Apr 07 '25 09:04 matyhtf

Hi. Please tell me what framework you are using. If possible, could you provide a reproducible code example? Thank you @jilcimar @ElielFC @b-bianca

The framework used is Hyperf, and here is the code snippet showing an example of the coroutine usage that is causing the issue:


co(function () use ($parameter1, $parameter2, $parameter3, $parameter4, $parameter5) {
    try {
        $client = new SnsClient(config("notification." . config('notification.requests_logs.config')));
        $message = [
            //content
        ];
        $client->publish([
            'TopicArn' => config('notification.requests_logs.topic'),
            'Subject' => 'Requests Logs',
            'Message' => json_encode($message),
        ]);
    } catch (Throwable $e) {
        logger()->error("{$e->getMessage()}, trace: {$e->getTraceAsString()}");
    }
});

I had to hide the contents of the $message array for confidentiality reasons.

jilcimar avatar Apr 07 '25 12:04 jilcimar

Could you please run php --ri curl? @jilcimar

NathanFreeman avatar Apr 07 '25 16:04 NathanFreeman

Could you please run php --ri curl? @jilcimar

Image

jilcimar avatar Apr 07 '25 17:04 jilcimar

@jilcimar Please provide the full namespace and Composer package name.

matyhtf avatar Apr 08 '25 02:04 matyhtf

I cannot replicate your issue. My curl version is 7.81.0, and my PHP version is 8.2.23.

Image

matyhtf avatar Apr 08 '25 08:04 matyhtf

I cannot replicate your issue. My curl version is 7.81.0, and my PHP version is 8.2.23.

@matyhtf My curl version is 8.12.1, and my PHP version is 8.3.15 .

Guzzle version is "guzzlehttp/guzzle": "^7.9" - 7.9.2 AWS SDK version is "aws/aws-sdk-php": "^3.324" - 3.339.14

It only happens in the production environment and when the request volume is high

jilcimar avatar Apr 08 '25 13:04 jilcimar

@jilcimar Please test with the latest code from the GitHub master branch, setting the print_backtrace_on_error option to true. This option will print the call stack when an unknown error occurs. This is an unknown error, and I cannot currently think of the circumstances under which this issue might arise.

matyhtf avatar Apr 14 '25 06:04 matyhtf

print_backtrace_on_error

@matyhtf This option is for enable on Swoole Server or swoole.ini file?

iamrafaelmelo avatar Apr 17 '25 18:04 iamrafaelmelo

@iamrafaelmelo

This is a global option that can be set as follows:

\Swoole\Coroutine::set(['print_backtrace_on_error' => true]);
// or 
swoole_async_set(['print_backtrace_on_error' => true]);

Thank you for your bug report. We need more information to analyze the issue. What impacts arise after this error log appears?

matyhtf avatar Apr 17 '25 23:04 matyhtf

I recommend installing the Boost Stacktrace library before compiling the new version of swoole.

sudo apt install libboost-stacktrace-dev

matyhtf avatar Apr 18 '25 09:04 matyhtf

@iamrafaelmelo

This is a global option that can be set as follows:

\Swoole\Coroutine::set(['print_backtrace_on_error' => true]); // or swoole_async_set(['print_backtrace_on_error' => true]); Thank you for your bug report. We need more information to analyze the issue. What impacts arise after this error log appears?

Sorry for long time without response (holidays), i'm deploying the service with this flag enable today, when i have some tests, will be back here.

iamrafaelmelo avatar Apr 22 '25 11:04 iamrafaelmelo

@matyhtf

I "enabled" these settings, but without success, maybe because Hyperf rewrote the settings. The most I could see was the new log format, that's all.

[2025-04-22 16:47:55 *12.3]	WARNING	ReactorEpoll::add() (ERRNO 801): failed to add events[fd=31#3, type=27, events=512]

PS: we are using the master Swoole

iamrafaelmelo avatar Apr 22 '25 20:04 iamrafaelmelo

@iamrafaelmelo Please execute the following command to update swoole-src and recompile, ensuring that the cache is cleared.

git pull
phpize
./configure [your options]
make clean
make -j 8
make install

Check the output of php --ri swoole or look at config.h to see if boost stacktrace or execinfo.h is enabled.

matyhtf avatar Apr 25 '25 03:04 matyhtf

@matyhtf I achieve install boost-dev on Alpine Linux, and works, on enable print_backtrace_on_error option we have been this results

Image

Image

Alpine

  • We are using the latest version (3.21)

Swoole informations

Image

PHP version and extensions

Image

iamrafaelmelo avatar May 12 '25 13:05 iamrafaelmelo

Any updates about this?

iamrafaelmelo avatar Aug 28 '25 12:08 iamrafaelmelo

I have encountered the same issue. Luckily, I was able to reproduce it in my local environment. Here is a minimal example to reproduce the error.

Steps to reproduce:

  • docker-compose up
  • docker run --rm -i --add-host=host.docker.internal:host-gateway grafana/k6 run - < test.js

Error WARNING ReactorEpoll::add() occurs in logs of service-a

curl_reproducer.zip

MartinKrizan avatar Sep 04 '25 17:09 MartinKrizan

docker run --rm -i --add-host=host.docker.internal:host-gateway grafana/k6 run - < test.js

Yes, reproduced locally:

Image

hydrapolic avatar Sep 19 '25 11:09 hydrapolic

I have encountered the same issue. Luckily, I was able to reproduce it in my local environment. Here is a minimal example to reproduce the error.

Steps to reproduce:

  • docker-compose up
  • docker run --rm -i --add-host=host.docker.internal:host-gateway grafana/k6 run - < test.js

Error WARNING ReactorEpoll::add() occurs in logs of service-a

curl_reproducer.zip

OK, I will try to reproduce this problem locally recently.

@MartinKrizan @hydrapolic

matyhtf avatar Sep 29 '25 02:09 matyhtf

Fixed. This is caused by two reasons.

The first problem is that the bottom layer of swoole supports both PHP7 and PHP8 through compatible code. Only std:: unordered_map<CURL *, Handle *> handle_buckets can store private data using curl_easy * pointer as the hashmap of the key. The latest PHP8 code does not use the CURLOPT_PRIVATE of libcurl for the implementation of the CURLOPT_PRIVATE option, but is implemented at the php level. This allows the underlying layer to use the CURLOPT_PRIVATE of libcurl to store the Handle pointer. The second problem is that there is a problem with the handling of the curl handle of keepalive. When curl multi remove the handle, must clear all socket fds of curl_easy. Otherwise, incorrect event listening will occur

matyhtf avatar Sep 29 '25 07:09 matyhtf

Thanks @matyhtf for fixing this, appreciated!

hydrapolic avatar Sep 29 '25 09:09 hydrapolic

Just tested with swoole-master (https://github.com/swoole/swoole-src/commit/8a56a771435711223baaff01266360a73e913f2d) and the warnings are gone 👍

hydrapolic avatar Oct 09 '25 15:10 hydrapolic