gunicorn
gunicorn copied to clipboard
benoitc
setproctitle causes segfaults in workers on macOS 12.6
This is related to the classic "fork() is unsafe without exec() on macOS" issue that's probably come up for gunicorn previously. In this current iteration, I was running apache airflow on macOS 12.6 and was experiencing reliable crashes (segfault, signal 11) when using python-ldap (with a native extension) in a worker process post-fork. On a cursory look in the gunicorn issue tracker, I see a number of issues that look like they could be people unknowingly stumbling upon this issue.
There is a long discussion of the issue from setproctitle's side here, as well as my exact environment/reproduction information on the issue I originally submitted to them here, before I realized quite what was going on.
As I understand it, the thing that's changed relatively recently is that now just the import of setproctitle is enough to make things crashy. I see that gunicorn has logic in place to attempt an import and fall back to using nothing. This would be fine for me if airflow did not require having the module installed, and I'm sure others have run into similar issues in the past. Some ideas to mitigate this:
- Config flag to disable attempting to import or use
setproctitleat all - Only import
setproctitlewhen it is used... should be a no-op on subsequent imports... it could be okay for use if its only imported postfork() - Default to disabling setproctitle on apple.
I know this issue is somewhat out of scope for gunicorn and in the thread of apple making things annoying for no reason, but I wanted to at least raise awareness of this issue. There are a lot of different modules/combinations with native extensions that could causes crashes in workers.
Is there any update for this issue? Or is there any work around I can do in order not to get those errors on my mac?
This is again an example for error:
[2023-10-04 09:20:36 +0300] [69944] [WARNING] Worker with pid 69968 was terminated due to signal 2
