Should kernel restart interrupt running code?

[jcrist]

I'm not sure if this is a bug or intended behavior. AFAICT, if a user restarts a kernel while the interpreter is actively processing, no SIGINT is sent to the interpreter process, and eventually the process is SIGKILL'd. This can lead to atexit handlers never running.

This is causing issues in dask, where we use atexit handlers to ensure remote resources are reclaimed.

Here's a reproducer. Paste this into a notebook cell, run, then restart the kernel before the cell finishes executing.

import atexit
import os
import time

with open("temp-file", "w") as f:
    f.write("This file should be deleted by the exit handler")

@atexit.register
def exit_handler():
    os.unlink("temp-file")
    
time.sleep(1000)

# Restart the kernel, `temp-file` should be deleted but isn't.
# If you interrupt, then restart, then `temp-file` is deleted

If you instead interrupt the kernel then restart it, everything is cleaned up properly. Is this intended behavior (and the issue is user workflow, rather than a possible bug)?

[jcrist]

Filed here since AFAICT this is kernel specific, rather than notebook related

[jcrist]

(cc @minrk if you have a second)

[damianavila]

Also, cc'ing some other people who could have some ideas too :wink:: @takluyver @Carreau @blink1073.

[Carreau]

That will likely be in Jupyter Client and we can likely try in order sigint/term/kill

[Carreau]

One of the questions will be the timeout for each.

[jcrist]

Ah, I wasn't sure if this was in jupyter_client or kernel specific. Should I refile there?

[Carreau]

No I think it's good here; though I think the patch if any would be in jupyter_client, as it is the one that currently sends signals.

[blink1073]

I agree with @Carreau, probably makes sense to add an interrupt call here using this logic.

[bilderbuchi]

I'm still reproducing this issue using ipykernel 6.2.0 and notebook 6.4.3 (on Windows) Actually, it seems to have grown worse, since the stated workaround

If you instead interrupt the kernel then restart it, everything is cleaned up properly.

does not work anymore. 😞

I suspect the interrupting operation does not actually complete. When I press "Interrupt kernel" I get the small "interrupting kernel" button for a second or two, which then disappears, but the Kernel indicator circle stays full (Kernel busy) and the tab icon remains an hourglass.

Has another workaround surfaced in the meantime? I have an application that spawns processes (with a cleanup procedure defined in __del__), and these processes seem to remain running (and eating memory) when I'm restarting the kernel. I have no atexit handler registered yet.