obelisk
obelisk copied to clipboard
ob run can leave a rogue GHC instance running indefinitely
Numerous people have reported that a ghc
process can be left behind after ob run
has been closed or even after a ghcid
reload.
Currently withProcess
in cliapp
doesn't actually kill the subprocess when it gets an async exception. Killing ob run
however does seem to kill subprocesses most of the time, but perhaps our sloppiness here is contributing.
The only way to kill this rogue ghc is to kill -9
.
Typically the rogue ghc is using a lot of CPU and memory.
I can confirm encountering this bug today. Do you think the withProcess
problem you describe above could be an easy fix?
I sort of doubt it will actually fix it unless we add more logic for "Hm you're taking a long time, I really want you to die so SIGKILL this time"
To clarify, what we definitely should be doing is bracket
ing our process creation properly in withProcess
and calling terminateProcess
in the "release". However, I've found that these rogue GHC's don't die with normal SIGTERM and must be SIGKILLed, which is something terminateProcess
doesn't do. So we'd have to further extend our bracketing to add a fallback to SIGKILL and that sounds a bit tricky to get right in general.
Even I have hit this issue many times recently. The rogue ghc needs to be terminated manually by kill -9
. And it takes up most of the system's memory.