DBdownload
DBdownload copied to clipboard
ldx
Filename case causing crash on Linux
I'm getting an error with DBdownload due to case mismatches. It looks like Dropbox returns lowercase directory names for all items, even in the metadata dict. Only the filenames are coming through with the proper casing. I'm not sure what a proper fix would be, but figured I'd bring it up here in case someone else comes across the same issue.
Here is what I hope is all the relevant info:
I am using DBdownload pulled straight from Git (Reports as v0.2, tree is at https://github.com/ldx/DBdownload/tree/62c269908a7f0177df585d5937e521ce554d1ab0 running on a Linux server:
Linux cloud 2.6.32-042stab092.3 #1 SMP Sun Jul 20 13:27:24 MSK 2014 x86_64 GNU/Linux
I'm calling the script like this:
dbdownload --source=toolkit --target=/var/www/toolkit
This is the traceback:
Traceback (most recent call last):
File "/home/jaquer/bin/dbdownload", line 9, in <module>
load_entry_point('dbdownload==0.2', 'console_scripts', 'dbdownload')()
File "build/bdist.linux-x86_64/egg/dbdownload/dbdownload.py", line 504, in main
File "build/bdist.linux-x86_64/egg/dbdownload/dbdownload.py", line 104, in start
File "build/bdist.linux-x86_64/egg/dbdownload/dbdownload.py", line 154, in _monitor
File "build/bdist.linux-x86_64/egg/dbdownload/dbdownload.py", line 307, in _apply_delta
File "build/bdist.linux-x86_64/egg/dbdownload/dbdownload.py", line 376, in _get_file
IOError: [Errno 2] No such file or directory: '/var/www/toolkit/rbtray/32bit/RBTray.exe'
This is the directory listing on my Windows 7 machine, where I added the directories/files:
D:\Cloud\Dropbox>tree /f toolkit
Folder PATH listing
Volume serial number is B4EF-4F81
D:\CLOUD\DROPBOX\TOOLKIT
│ PStart.exe
│
├───RBTray
│ ├───32bit
│ │ RBHook.dll
│ │ RBTray.exe
│ │
│ └───64bit
│ RBHook.dll
│ RBTray.exe
│
└───_cleanup
Finally, I ran the script using pdb to see if I could figure out the problem. As far as I can tell, the problem is that the client.delta() call on here: https://github.com/ldx/DBdownload/blob/62c269908a7f0177df585d5937e521ce554d1ab0/dbdownload/dbdownload.py#L137 is returning this result (cursor removed for brevity):
{
"cursor": "DEADBEEF",
"entries": [
[
"/toolkit/_cleanup",
{
"bytes": 0,
"icon": "folder",
"is_dir": true,
"modified": "Mon, 29 Sep 2014 06:48:45 0000",
"path": "/toolkit/_cleanup",
"rev": "54246e29000cdbcf",
"revision": 1411673641,
"root": "dropbox",
"size": "0 bytes",
"thumb_exists": false
}
],
[
"/toolkit/_cleanup/test",
null
],
[
"/toolkit/_cleanup/movefile.exe",
null
],
[
"/toolkit/new folder",
null
],
[
"/toolkit/rbtray",
{
"bytes": 0,
"icon": "folder",
"is_dir": true,
"modified": "Mon, 29 Sep 2014 06:51:07 0000",
"path": "/toolkit/RBTray",
"rev": "54246e37000cdbcf",
"revision": 1411673655,
"root": "dropbox",
"size": "0 bytes",
"thumb_exists": false
}
],
[
"/toolkit/rbtray/64bit",
{
"bytes": 0,
"icon": "folder",
"is_dir": true,
"modified": "Mon, 29 Sep 2014 06:51:27 0000",
"path": "/toolkit/RBTray/64bit",
"rev": "54246e3a000cdbcf",
"revision": 1411673658,
"root": "dropbox",
"size": "0 bytes",
"thumb_exists": false
}
],
[
"/toolkit/rbtray/32bit",
{
"bytes": 0,
"icon": "folder",
"is_dir": true,
"modified": "Mon, 29 Sep 2014 06:51:27 0000",
"path": "/toolkit/RBTray/32bit",
"rev": "54246e3b000cdbcf",
"revision": 1411673659,
"root": "dropbox",
"size": "0 bytes",
"thumb_exists": false
}
],
[
"/toolkit/rbtray/32bit/rbtray.exe",
{
"bytes": 39936,
"client_mtime": "Sun, 30 Oct 2011 22:02:44 0000",
"icon": "page_white_gear",
"is_dir": false,
"mime_type": "application/x-msdos-program",
"modified": "Mon, 29 Sep 2014 06:51:32 0000",
"path": "/toolkit/rbtray/32bit/RBTray.exe",
"rev": "54246e40000cdbcf",
"revision": 1411673664,
"root": "dropbox",
"size": "39 KB",
"thumb_exists": false
}
],
[
"/toolkit/rbtray/32bit/rbhook.dll",
{
"bytes": 33792,
"client_mtime": "Sun, 30 Oct 2011 22:02:40 0000",
"icon": "page_white_gear",
"is_dir": false,
"mime_type": "application/x-msdos-program",
"modified": "Mon, 29 Sep 2014 06:51:32 0000",
"path": "/toolkit/rbtray/32bit/RBHook.dll",
"rev": "54246e41000cdbcf",
"revision": 1411673665,
"root": "dropbox",
"size": "33 KB",
"thumb_exists": false
}
],
[
"/toolkit/rbtray/64bit/rbtray.exe",
{
"bytes": 45568,
"client_mtime": "Sun, 30 Oct 2011 22:02:24 0000",
"icon": "page_white_gear",
"is_dir": false,
"mime_type": "application/x-msdos-program",
"modified": "Mon, 29 Sep 2014 06:51:32 0000",
"path": "/toolkit/rbtray/64bit/RBTray.exe",
"rev": "54246e42000cdbcf",
"revision": 1411673666,
"root": "dropbox",
"size": "44.5 KB",
"thumb_exists": false
}
],
[
"/toolkit/rbtray/64bit/rbhook.dll",
{
"bytes": 38912,
"client_mtime": "Sun, 30 Oct 2011 22:02:20 0000",
"icon": "page_white_gear",
"is_dir": false,
"mime_type": "application/x-msdos-program",
"modified": "Mon, 29 Sep 2014 06:51:32 0000",
"path": "/toolkit/rbtray/64bit/RBHook.dll",
"rev": "54246e43000cdbcf",
"revision": 1411673667,
"root": "dropbox",
"size": "38 KB",
"thumb_exists": false
}
]
],
"has_more": false,
"reset": false
}
Let me know if you have any questions.
Good catch, I think I've seen a similar issue when synchronizing my files from Dropbox. At that time I just changed the directory name.
I'm not sure when I'll have the time to fix this. PRs are welcome though. :)
Could maybe DBdownload handle everything in lowercase on the local directory?
This is an issue even with official (zip) downloads.
If you are to download a whole directory from a dropbox remote on linux the file will be unzipped with mismatched filename cases without double -L flag.
In above case the fix is to do unzip -LL which lowercases everything, maybe we could use a similar flag?
The problem will be of course then that synced files on linux will always be lowercase but I think that's better than it not working at all currently (when you can't rename the remote because you only have read-only permissions)
I've just added .lower() as workaround.
Spoke too soon, merely doing:
--- upstream/dbdownload/dbdownload.py 2018-11-10 02:42:10.868275729 +0000
+++ DBdownload/lib/python2.7/site-packages/dbdownload/dbdownload.py 2018-11-10 02:46:01.851601168 +0000
@@ -144,7 +144,7 @@
if not remote_comp[-1]: # Trailing slash.
rootlen -= 1
x = remote.split(dropboxpath.sep)[rootlen:]
- local = os.path.join(self.local_dir, *x)
+ local = os.path.join(self.local_dir, *x).lower()
return local
def _monitor(self):
to treat local files as lowercase is not enough as dropbox lib errors when cache is loaded next start:
2018-11-10 02:47:36,407 *** DBdownload v0.3 starting up ***
2018-11-10 02:47:36,407 loaded token
2018-11-10 02:47:36,407 loaded delta cursor
2018-11-10 02:47:36,580 loaded local tree
Traceback (most recent call last):
File "DBdownload/bin/dbdownload", line 11, in <module>
load_entry_point('dbdownload==0.3', 'console_scripts', 'dbdownload')()
File "/home/tete/code/dboxsyncgit/DBdownload/lib/python2.7/site-packages/dbdownload/dbdownload.py", line 529, in main
dl.start()
File "/home/tete/code/dboxsyncgit/DBdownload/lib/python2.7/site-packages/dbdownload/dbdownload.py", line 118, in start
self._monitor()
File "/home/tete/code/dboxsyncgit/DBdownload/lib/python2.7/site-packages/dbdownload/dbdownload.py", line 156, in _monitor
changed = self._check_missing()
File "/home/tete/code/dboxsyncgit/DBdownload/lib/python2.7/site-packages/dbdownload/dbdownload.py", line 277, in _check_missing
local_path = unicode(self._remote2local(meta.path_display))
File "/home/tete/code/dboxsyncgit/DBdownload/lib/python2.7/site-packages/dropbox/files.py", line 2400, in path_display
if self._path_display_present:
AttributeError: _path_display_present