obelisk icon indicating copy to clipboard operation
obelisk copied to clipboard

ob run can leave a rogue GHC instance running indefinitely

Open 3noch opened this issue 5 years ago • 7 comments

Numerous people have reported that a ghc process can be left behind after ob run has been closed or even after a ghcid reload.

3noch avatar Jan 30 '20 20:01 3noch

Currently withProcess in cliapp doesn't actually kill the subprocess when it gets an async exception. Killing ob run however does seem to kill subprocesses most of the time, but perhaps our sloppiness here is contributing.

3noch avatar Jan 30 '20 20:01 3noch

The only way to kill this rogue ghc is to kill -9.

3noch avatar Jan 30 '20 20:01 3noch

Typically the rogue ghc is using a lot of CPU and memory.

3noch avatar Jan 30 '20 20:01 3noch

I can confirm encountering this bug today. Do you think the withProcess problem you describe above could be an easy fix?

maralorn avatar Mar 31 '20 15:03 maralorn

I sort of doubt it will actually fix it unless we add more logic for "Hm you're taking a long time, I really want you to die so SIGKILL this time"

3noch avatar Mar 31 '20 20:03 3noch

To clarify, what we definitely should be doing is bracketing our process creation properly in withProcess and calling terminateProcess in the "release". However, I've found that these rogue GHC's don't die with normal SIGTERM and must be SIGKILLed, which is something terminateProcess doesn't do. So we'd have to further extend our bracketing to add a fallback to SIGKILL and that sounds a bit tricky to get right in general.

3noch avatar Mar 31 '20 20:03 3noch

Even I have hit this issue many times recently. The rogue ghc needs to be terminated manually by kill -9. And it takes up most of the system's memory.

dfordivam avatar Feb 23 '21 12:02 dfordivam