SCOOP hangs on attempting to read remote stdout (on stable version - 0.7, r1.1)

[Carnou]

Using python 3.5.1 and the latest stable version of SCOOP (i.e. the one available through pip - version 0.7, revision 1.1), any time I run any of the example files, e.g. python -m scoop -vv --hostfile hostfile rssDoc.py, it hangs at the end. I see a message such as "Closing workers on solomon (8 workers)", and nothing more. Debugging through pdb showed me that it was blocking as it tried to read the process.stdout in the close function of workerLaunch.py.

I solved this by making a function to make the stream non-blocking before reading it, and used that on both process.stdout and process.stderr:

import fcntl
import os

def _makeStreamNonBlocking(self, stream):
    flags = fcntl.fcntl(stream.fileno(), fcntl.F_GETFL)
    fcntl.fcntl(stream.fileno(), fcntl.F_SETFL, flags | os.O_NDELAY)

def close(self):
         """Connection(s) cleanup."""
         # Ensure everything is cleaned up on exit
         scoop.logger.debug('Closing workers on {0}.'.format(self))

         # Output child processes stdout and stderr to console
         for process in self.subprocesses:
             if process.stdout is not None:
                 self._makeStreamNonBlocking(process.stdout)
                 sys.stdout.write(process.stdout.read().decode("utf-8"))
                 sys.stdout.flush()

             if process.stderr is not None:
                 self._makeStreamNonBlocking(process.stderr)
                 sys.stderr.write(process.stderr.read().decode("utf-8"))
                 sys.stderr.flush()

I see this code I modified doesn't even exist in the current version on github. Should I simply favor the github version over the "currently stable" one?

[soravux]

This section has changed since the stable version. Does it happens with the development version (the master branch)? If so, please keep me posted.