php
Segmentation fault during runtime [#1841]
@AlliBalliBaba
What happened?
{"level":"warn","ts":1756707392.6662247,"logger":"tls","msg":"stapling OCSP","identifiers":["my.server.>
{"level":"error","ts":1756720415.2692719,"logger":"frankenphp","msg":"PHP Fatal error: Allowed memory size >
{"level":"error","ts":1756720415.2759776,"logger":"frankenphp","msg":"PHP Fatal error: Allowed memory size >
{"level":"error","ts":1756720426.528808,"logger":"frankenphp","msg":"PHP Fatal error: Allowed memory size o>
{"level":"error","ts":1756720426.5313344,"logger":"frankenphp","msg":"PHP Fatal error: Allowed memory size >
{"level":"error","ts":1756720439.5222485,"logger":"frankenphp","msg":"PHP Fatal error: Allowed memory size >
0425]: [🡕] Process 40011 (frankenphp) of user 982 dumped core.
Stack trace of thread 40034:
#0 0x000055949c4ad201 n/a (n/a + 0x0)
#1 0x000055949c483c27 n/a (n/a + 0x0)
#2 0x000055949c4827ad n/a (n/a + 0x0)
#3 0x000055949c4ad509 n/a (n/a + 0x0)
#4 0x00007f8856068070 __restore_rt (libc.so.6 + 0x41070)
#5 0x00007f88561839c8 __memcpy_avx_unaligned_erms (libc.so.6 + 0x15c9c8)
#6 0x00007f8856eaae43 zend_stack_push (libphp-zts-84.so + 0x8aae43)
#7 0x00007f8856d2502d php_output_handler_start (libphp-zts-84.so + 0x72502d)
#8 0x00007f8856d269f1 php_output_start_user (libphp-zts-84.so + 0x7269f1)
#9 0x00007f8856d271ec n/a (libphp-zts-84.so + 0x7271ec)
#10 0x00007f8856e4d7ca execute_ex (libphp-zts-84.so + 0x84d7ca)
#11 0x00007f8856debe75 zend_call_function (libphp-zts-84.so + 0x7ebe75)
#12 0x00007f8856c75020 n/a (libphp-zts-84.so + 0x675020)
#13 0x00007f8856e607b3 zend_hash_apply (libphp-zts-84.so + 0x8607b3)
#14 0x00007f8856c78b09 php_call_shutdown_functions (libphp-zts-84.so + 0x678b09)
#15 0x00007f8856d1f5bd php_request_shutdown (libphp-zts-84.so + 0x71f5bd)
#16 0x000055949d93d546 n/a (n/a + 0x0)
#17 0x000055949d93d62d n/a (n/a + 0x0)
#18 0x00007f88560bede5 start_thread (libc.so.6 + 0x97de5)
#19 0x00007f885613198c __clone3 (libc.so.6 + 0x10a98c)
Stack trace of thread 40015:
#0 0x000055949c4ad7a3 n/a (n/a + 0x0)
#1 0x000055949c43be07 n/a (n/a + 0x0)
#2 0x000055949c471d13 n/a (n/a + 0x0)
#3 0x000055949c47437a n/a (n/a + 0x0)
#4 0x000055949c474865 n/a (n/a + 0x0)
#5 0x000055949c4a9957 n/a (n/a + 0x0)
ELF object binary architecture: AMD x86-64
Build Type
Custom (tell us more in the description)
Worker Mode
No
Operating System
GNU/Linux
CPU Architecture
x86_64
PHP configuration
[[email protected] fld]# frankenphp version
FrankenPHP v1.9.1 PHP 8.4.12 Caddy v2.10.2 h1:g/gTYjGMD0dec+UgMw8SnfmJ3I9+M2TdvoRL/Ovu6U8=
Relevant log output
-
Yeah looks the same. From the output it also might be happening on worker restart (an unclean shutdown?). Are you using any framework/special library? How many threads are you spawning?
Would be nice if this could be made reproducible somehow.
A symfony app with 16 worker threads on this host. Nothing special going on, haven't encountered this before 1.9.0. (Which doesn't necessarily mean it's a regression, I just never noticed it before).
I wish I had a reproducer, but I don't know when exactly it happens. The last time may have happened while no requests were coming in.
The only prominent change I can think of was moving symfony messenger workers from crontab to caddy-supervisor some time around v1.7.0 or v1.8.0.
How are you running the messengers? Via php or frankenphp php-cli? Are they running in the same process as the web server?
Will have a closer look once I got time
Looking at the output again it's probably related to something happening in a function registered with register_shutdown_function. This might also be happening in php-cli, not frankenphp.
The only thing I could find out from a little bit of testing is that doing something like this will cause a small memory leak:
register_shutdown_function(function () {
ob_start();
});
That's at least what it looks like from the output: a shutdown-function calling php_output_start_user (ob_start).
Can you maybe look up if any library you are using is doing something like this?
The only library I've recently added was Sentry. I'll check if any such thing is used there.
The php-cli jobs are just symfony messenger and scheduler. Hard to imagine something being wrong with those.
Sentry does indeed use register_shutdown_function in two places, once in its own source code, once in the required react/event-loop dependency.
Yeah this might be sentry trying to handle a fatal error here, hard to say though from just the output.
The error handler might create a new output buffer.
I'll switch my dev environment over to a version with debugging symbols left in. Unfortunately it might not happen for a long time, I've scoured my logs and found nothing there.
I don't think it's related to the supervisor running php-cli though, as my understanding is that it creates extra processes for that. One of them failing wouldn't crash the webserver (and therefore the service).
For what it's worth, it still hasn't happened again. This may take a while to find a reproducer...
Got it again, but in production this time...
I'm working on adding debuginfo packages, hopefully for 1.9.2.