holodeck icon indicating copy to clipboard operation
holodeck copied to clipboard

Holodeck binaries not dying

Open joshgreaves opened this issue 5 years ago • 2 comments

I've been building an agent in holodeck, and noticed that after a while I couldn't load it any more. I killed my python script each time, but when I run nvidia-smi I see:

+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID   Type   Process name                             Usage      |
|=============================================================================|
|    0      1685      G   /usr/lib/xorg/Xorg                            38MiB |
|    0      1797      G   /usr/bin/gnome-shell                          10MiB |
|    0      2185      G   /usr/lib/xorg/Xorg                           380MiB |
|    0      2346      G   /usr/bin/gnome-shell                         304MiB |
|    0      2949      G   ...uest-channel-token=17994277894646646720   305MiB |
|    0      3645      G   ...-token=63BF3768D015AB609FC150798E4990A8   109MiB |
|    0      6433      G   ...opt/mendeleydesktop/bin/mendeleydesktop    11MiB |
|    1      6876    C+G   ...Editor/Holodeck/Binaries/Linux/Holodeck   275MiB |
|    1      9393    C+G   ...Editor/Holodeck/Binaries/Linux/Holodeck   275MiB |
|    1     11763    C+G   ...Editor/Holodeck/Binaries/Linux/Holodeck   275MiB |
|    1     11941    C+G   ...Editor/Holodeck/Binaries/Linux/Holodeck   275MiB |
|    1     12015    C+G   ...Editor/Holodeck/Binaries/Linux/Holodeck   275MiB |
|    1     12093    C+G   ...Editor/Holodeck/Binaries/Linux/Holodeck   275MiB |
+-----------------------------------------------------------------------------+

top:

12093 josh      20   0 1982876 443952 158308 S  15.9  0.7   0:27.32 Holodeck    
11763 josh      20   0 1917440 361592 155468 S  15.6  0.5   0:33.72 Holodeck    
11941 josh      20   0 1917408 361592 155596 S  15.6  0.5   0:26.78 Holodeck    
 6876 josh      20   0 1961420 379364 155948 S  15.3  0.6   9:28.53 Holodeck    
 9393 josh      20   0 1961432 369632 155788 S  15.3  0.6   5:08.93 Holodeck    
12015 josh      20   0 1982800 443924 158104 S  15.3  0.7   0:28.71 Holodeck    

For some reasons the binaries stick around even though the window disappears.

joshgreaves avatar Mar 25 '19 21:03 joshgreaves

Could be something to do with this. My script keeps dying with signal 7, it could be that this signal doesn't get forwarded to Holodeck Process finished with exit code 135 (interrupted by signal 7: SIGEMT)

joshgreaves avatar Mar 25 '19 22:03 joshgreaves

How are you running/stopping the scripts?

Depending on how this exception is happening, Holodeck might not have the chance to kill the engine before the interpreter exits. I would suspect that those are zombie processes at this point, since the parent (python interpreter) died.

Using a process group might help.

jaydenmilne avatar Mar 26 '19 16:03 jaydenmilne