chuck.exe hangs on kill signal from windows/powershell with mouse input in command prompt/powershell

[nshaheed]

If you run examples/mouse.ck from cmd/powershell in windows and then try to kill with ctrl+c, you get the message `[chuck]: cleaning up...' and then it hangs and you can't escape. The kill signal works fine with git bash (which is a bash-style shell)

I have two working theories right now

• ctrl+c on cmd/powershell might be sending some weird windows-specific signal (like line-endings being different), which would need to be caught
• this may be related to #200 potentially merging that would solve the issue or provide a way forward?

[nshaheed]

I also had this happen in the WSL 2 shell (with ubuntu)

[dbadb]

Hey Nick, I believe this new datapoint may be relevant. I have a growing suspicion that there's a problem with the global_manager. We might need to get @lathertonj to chime in here.

Here's a simple script

global float xx;
0 => xx;

for(int i;i<10;i++)
{
    1::second => now;
    <<<"chuck global xx", xx>>>;
}

Run this in a windows-debugging build and ctrl-c before 10 sec. I'm using git-bash, fwiw.

[chuck]: cleaning up...
Assertion failed: m_ref_count > 0, file core/chuck_oo.cpp, line 129

Now comment out the assignment 0 => xx and try again. For me no crash. It's as though assigning to the global grabs a reference of some sort...

Now, when I attach a debugger (windows, vs17) I get this stack trace

chuck.exe!type_engine_shutdown(Chuck_Env * env) Line 482
chuck.exe!Chuck_Compiler::shutdown() Line 173
chuck.exe!Chuck_Compiler::~Chuck_Compiler() Line 107
[External Code]
chuck.exe!ChucK::shutdown() Line 792
chuck.exe!ChucK::~ChucK() Line 164
[External Code]	
chuck.exe!global_cleanup() Line 299
chuck.exe!main(int argc, const char * * argv) Line 133

and here's the body of type_engine_shutdown where the contents of env seem dubious (as though someone else has already deleted it).

void type_engine_shutdown( Chuck_Env * env )
{
    // log
    EM_log( CK_LOG_SEVERE, "shutting down type checker..." );
    
    // shut it down
    SAFE_DELETE( env );   // <----- occasional crash here
    
    // log
    EM_log( CK_LOG_SEVERE, "type checker shutdown complete." );
}

[dbadb]

Btw: it's possible that the Event-system teardown is a separate but related issue. My guess is that the global shutdown triggered by ctrl-c is at odds with some other shutdown code and we're getting > 1 deletes. This may be more evident on windows-debug builds because they instrument debugging builds with a variety of memory-overrun and double-free fences to help trigger lurking badness.

[nshaheed]

Iteresting... I don't know the system well enough to figure this out without doing a really deep dive into things, but I tried recreating the crash on osx using your simple script with the bash script below. I had it running for around 2 hours and didn't get any crashes so this is either a windows-specific thing or it's really rare on other os's (also, I forgot to mention in my above comments that chuck crashed in a non-debug build in WSL2, which supports your theory that this is about multiple deletes)

while true
do
# Launch script in background                                                                                                                                                                               
./chuck global-crash.ck &
# Get its PID                                                                                                                                                                                               
PID=$!

# sleeptime is somewhere between 0 and 10 seconds                                                                                                                                                           
sleeptime=$(awk -v seed="$RANDOM" 'BEGIN { srand(seed); printf("%.4f\n", 10 * rand()) }')

echo $sleeptime
sleep $sleeptime
# Kill it                                                                                                                                                                                                   
kill -INT $PID

wait $PID
echo "$PID finished"                                                                                                                                                                                        
done

[dbadb]

Here's another clue. I notice that on windows the sigint is delivered in a separate thread from the Main thread. There seems to be a race condition between cleanups. Still haven't nailed it, but it's possible this accounts for the bad windows-specific behavior. Might be worth verifying which thread the signal is delivered to on linux and osx.

edit: I just verified on M1/MaxOS that the signal is delivered to the main thread so that would account for why we don't see this problem on mac.

[dbadb]

I put a print statement in ~ChucK and this log shows that it's being called twice (Destroy Chuck begin). Both of these calls are triggered by chuck_main::global_cleanup.

chuck tick 0.000000
[chuck]: cleaning up...
chuck_main::global_cleanup  begin --
[chuck]:(3:SEVERE): stopping real-time watch dog process...
[chuck]:(5:INFORM): detaching all resources...
[chuck]:(5:INFORM):  | shutting down HID...
[chuck]:(5:INFORM):  | shutting down serial devices
Destroy Chuck begin
[chuck]:(2:SYSTEM): shutting down ChucK instance...
[chuck]:(5:INFORM): cleaning up STK resources...
[chuck]:(5:INFORM):  | (via STK): detaching file handles...
[chuck]:(5:INFORM):  | (via STK): stopping write threads...
[chuck]:(2:SYSTEM): unprotecting special objects...
[chuck]:(2:SYSTEM): locking down special objects...
[chuck]:(2:SYSTEM): shutting down virtual machine...
[chuck]:(2:SYSTEM):  | unprotecting special objects...
[chuck]:(2:SYSTEM):  | freeing shreduler...
[chuck]:(2:SYSTEM):  | unregistering VM from HID manager...
[chuck]:(2:SYSTEM):  | unregistering VM from MIDI manager...
[chuck]:(2:SYSTEM):  | freeing msg/reply/event buffers...
[chuck]:(3:SEVERE):  | clearing shreds...
[chuck]:(3:SEVERE):  |  | freeing dumped shreds...
[chuck]:(2:SYSTEM):  | freeing special ugens...
[chuck]:(2:SYSTEM):  | locking down special objects...
[chuck]:(2:SYSTEM):  | virtual machine shutdown complete.
[chuck]:(2:SYSTEM): chuck_main::global_cleanup  begin --
Destroy Chuck begin
shutting down compiler...
[chuck]:(3:SEVERE):  | shutting down type checker...
[chuck]:(2:SYSTEM):  | shutting down ChucK instance...
[chuck]:(5:INFORM):  | cleaning up STK resources...
[chuck]:(5:INFORM):  |  | (via STK): detaching file handles...
[chuck]:(5:INFORM):  |  | (via STK): stopping write threads...
[chuck]:(2:SYSTEM):  | unprotecting special objects...
[chuck]:(2:SYSTEM):  | locking down special objects...
[chuck]:(2:SYSTEM):  | shutting down compiler...
[chuck]:(3:SEVERE):  |  | shutting down type checker...
[chuck]zsh: segmentation fault  chuck --verbose:5 bug.ck

[dbadb]

So here's a hack solution. In chuck_main.cpp::global_cleanup()

void global_cleanup()
{
    static int inprogress = 0;
    if(inprogress) return;
    inprogress = 1;

This eliminated the crashes (so far). Of course there's a more correct way to do this inprogress hack that involves mutexes or at least atomics.