dispy icon indicating copy to clipboard operation
dispy copied to clipboard

Error on dependency file transfer if starting from a windows root directory

Open davet2001 opened this issue 8 years ago • 3 comments

I have found that there is a small issue when transferring dependency files if the current working directory for a dispy client python script is a root directory (e.g. c:).

I was getting file not found errors exceptions raised during jobs on the server, and then when I turned on server debugging messages it turned out that files were being copied to the wrong location.

What I did Running from c:\ and I set c:\folder1\file.txt as a dependency...

What I expected to happen files copied to \folder1\file1.txt

What actually happened files copied to \older1\file.txt (note the missing f).

It was quite consistently truncating the first character.

Eventually I discovered that there is a small problem with the logic in init.py:DispyJob:init os.path.dirname(name[len(cwd+os.sep):]) evaluates to 'older1\file.txt' in this scenario, when evaluated on the client.

The problem is that cwd is c:\ which already contains a trailing slash. Adding the os.sep moves the slice point so you lose some of the filename.

Proposed patch:

 py2/dispy/__init__.py | 5 ++++-
 py3/dispy/__init__.py | 5 ++++-
 2 files changed, 8 insertions(+), 2 deletions(-)

diff --git a/py2/dispy/__init__.py b/py2/dispy/__init__.py
index c505dc1..579f6da 100644
--- a/py2/dispy/__init__.py
+++ b/py2/dispy/__init__.py
@@ -567,7 +567,10 @@ class _DispyJob_(object):
                 else:
                     name = os.path.abspath(dep)
                     if name.startswith(cwd):
-                        dst = os.path.dirname(name[len(cwd+os.sep):])
+                        if cwd.endswith(os.sep):
+                            dst = os.path.dirname(name[len(cwd):])
+                        else:
+                            dst = os.path.dirname(name[len(cwd+os.sep):])
                     else:
                         dst = '.'
                 if name in depend_ids:
diff --git a/py3/dispy/__init__.py b/py3/dispy/__init__.py
index de96a13..a712595 100755
--- a/py3/dispy/__init__.py
+++ b/py3/dispy/__init__.py
@@ -567,7 +567,10 @@ class _DispyJob_(object):
                 else:
                     name = os.path.abspath(dep)
                     if name.startswith(cwd):
-                        dst = os.path.dirname(name[len(cwd+os.sep):])
+                        if cwd.endswith(os.sep):
+                            dst = os.path.dirname(name[len(cwd):])
+                        else:
+                            dst = os.path.dirname(name[len(cwd+os.sep):])
                     else:
                         dst = '.'
                 if name in depend_ids:

davet2001 avatar Jan 10 '17 22:01 davet2001

Added patch file. dispy_windows_file_transfer_patch.txt

davet2001 avatar Jan 10 '17 22:01 davet2001

Committed an alternate fix. I haven't tested it, but hopefully it works. Please confirm.

pgiri avatar Jan 11 '17 01:01 pgiri

Can you confirm if current release (4.7.3) works or not?

pgiri avatar Apr 01 '17 19:04 pgiri