dispy
dispy copied to clipboard
Invalid syntax in dispynode.py
Syntax does not work with Python 2.7

Which version of Python are you using?
This can be fixed by replacing lines 880 and 881 with:
for var in globals():
if var not in self.__init_globals:
compute.globals[var] = globals()[var]
I've not getting that issue anymore, but I am now getting a new issue after having a cluster finish a certain amount of parallel dispy job iterations. (240 jobs for parallel submissions of 8 submitted and completed at a time.) [messages printed with dispynode.py -d and loglevel at DEBUG]
2018-08-23 20:00:29 dispy - Could not connect to 128.84.48.150:51348, Traceback (most recent call last):
File "/work/zhang/users/jg925/datuner-ml/releases/Linux_x86_64/install/lib/python2.7/site-packages/dispy/__init__.py", line 585, in send
resp = yield sock.recv_msg()
File "/work/zhang/users/jg925/datuner-ml/releases/Linux_x86_64/install/lib/python2.7/site-packages/pycos/__init__.py", line 3609, in _schedule
retval = task._generator.throw(*exc)
File "/work/zhang/users/jg925/datuner-ml/releases/Linux_x86_64/install/lib/python2.7/site-packages/pycos/__init__.py", line 868, in _async_recv_msg
data = yield self.recvall(n)
timeout: timed out
2018-08-23 20:00:29 dispy - Failed to run 140372617047888 on 128.84.48.150: Traceback (most recent call last):
File "/work/zhang/users/jg925/datuner-ml/releases/Linux_x86_64/install/lib/python2.7/site-packages/dispy/__init__.py", line 585, in send
resp = yield sock.recv_msg()
File "/work/zhang/users/jg925/datuner-ml/releases/Linux_x86_64/install/lib/python2.7/site-packages/pycos/__init__.py", line 3609, in _schedule
retval = task._generator.throw(*exc)
File "/work/zhang/users/jg925/datuner-ml/releases/Linux_x86_64/install/lib/python2.7/site-packages/pycos/__init__.py", line 868, in _async_recv_msg
data = yield self.recvall(n)
timeout: timed out
Th second issue is not related to first, so would've been better to post as new issue; I still would like to know if there is an issue with syntax in dispynode (seems to be with Windows, although second issue seems to be with Linux).
Second issue may be due to network or other issues? 'timed out' implies data could not be sent / received before time out. If you set reentrant=True (default is False) when creating cluster, jobs will be automatically (re) sent to another available node (however, re-entrant should be set only if computation can be re-evaluated without affecting your program's state). There is also recover_jobs function to retrieve job results in case client lost connection.
I am no longer getting a syntax error with dispynode. I will try applying your suggestions.