vmprof-python icon indicating copy to clipboard operation
vmprof-python copied to clipboard

fork() blocks gzip file descriptor from being closed

Open jonashaag opened this issue 9 years ago • 3 comments

Problem: If a Python program forks(), which makes a copy of all file descriptors, the child processes may block the gzip stdin file descriptor from being closed, which leads to vmprof never terminating.

Possible patch:

diff --git a/src/vmprof_main.h b/src/vmprof_main.h
index cf0b779..5c7a75d 100644
--- a/src/vmprof_main.h
+++ b/src/vmprof_main.h
@@ -250,6 +250,11 @@ static void atfork_enable_timer(void) {
     }
 }
 
+static void atfork_close_profile_file(void) {
+    if (profile_file != -1)
+        close(profile_file);
+}
+
 static int install_pthread_atfork_hooks(void) {
     /* this is needed to prevent the problems described there:
          - http://code.google.com/p/gperftools/issues/detail?id=278
@@ -263,7 +268,7 @@ static int install_pthread_atfork_hooks(void) {
     */
     if (atfork_hook_installed)
         return 0;
-    int ret = pthread_atfork(atfork_disable_timer, atfork_enable_timer, NULL);
+    int ret = pthread_atfork(atfork_disable_timer, atfork_enable_timer, atfork_close_profile_file);
     if (ret != 0)
         return -1;
     atfork_hook_installed = 1;

This simply closes the profile file fds in the child processes. I'm not sure we don't need them there though.

jonashaag avatar Nov 23 '16 23:11 jonashaag

Thanks, I applied the fix. Did you by chance test it on coala?

planrich avatar Nov 25 '16 09:11 planrich

Yes I tested it with coala only and it works

jonashaag avatar Nov 25 '16 10:11 jonashaag

The file descriptor is not piped to a gzip process anymore. Things got complicated with native profiling.

Now we need (just before vmprof is disabled) reiterate the profile to find the native addresses that must be logged to the profile. Having it piped to another process (and gzipped) makes that very hard.

I'll leave this ticket open until I'll add the compression back (just before the profile is uploaded).

planrich avatar Feb 14 '17 16:02 planrich