dropbox-sdk-python icon indicating copy to clipboard operation
dropbox-sdk-python copied to clipboard

Expose way to keep track of a file's upload or download progress

Open skela opened this issue 9 years ago • 12 comments

Having an optional delegate method argument in the files_upload / files_download methods so that we can update progress UI and whatnot when uploading and downloading files would be really useful.

skela avatar Dec 16 '15 04:12 skela

This is possibly common enough to add special support. But for now, there are some workarounds.

files_download should be easy. It returns the file metadata object and the HTTP response object (from the Requests library). The file metadata object tells you how big the file is. The HTTP response object's raw field is a file-like object, so you can track progress as you repeatedly call read(4096).

files_upload is a little more work: take your original file-like data source, wrap it with your own object, and pass that to files_upload. When files_upload calls your wrapper's read(), you can assume all data read before that point has been uploaded (and perhaps call a delegate).

The assumption (that all the data read before the call to read() has been uploaded) isn't bulletproof, but it might be good enough. The upside is that these wrappers are general-purpose.

cakoose avatar Dec 16 '15 06:12 cakoose

In the v1 API, the upload_chunk method returned an offset value - which is essentially the number of bytes uploaded. I use this for a progress bar in a CLI app at present. Does something of the sort exist in v2?

mafrosis avatar Dec 16 '15 06:12 mafrosis

v2 has upload_session_start / upload_session_append / upload_session_finish, but its not exactly for this use-case I don't think. Besides there's still no progress update mechanism for download, so would certainly be useful to get that in there, shouldn't be hard at all.

skela avatar Dec 16 '15 06:12 skela

:+1: Yeah it should be straight forward enough. Awesome job on opening the client SDK on github - I have a fork of v1 with a single lonely patch which nowadays will be a pull request :smile:

mafrosis avatar Dec 16 '15 06:12 mafrosis

@cakoose thanks for the workaround, I'll take a look at doing that for now.

skela avatar Dec 16 '15 07:12 skela

@cakoose your workaround is better than nothing, thanks!

skela avatar Dec 17 '15 04:12 skela

@cakoose I am confused in using the return type of files_download_to_file. as per the documentation dropbox sdk

The return types are metadata and request object. They are returned only after the file gets downloaded. I don't know how to invoke the read(4096) in the returned request object.
Maybe I don't know how to use this.

Moreover in this case, tuple is returned. But when printing it, I find only the metadata. screen shot 2015-12-23 at 3 29 51 pm and the code for that piece is screen shot 2015-12-23 at 3 31 07 pm

  • So, may be a small snippet on how to use read on the request object would be appreciated
  • In case of exceptions, like network time out, how to re-run it.( in simple a robust one)

Thanks in advance.

lakshmantgld avatar Dec 23 '15 06:12 lakshmantgld

Use files_download instead of files_download_to_file.

cakoose avatar Dec 23 '15 07:12 cakoose

@Lakshman-LD function files_download_to_file have mistake in their doc description. really returns only FileMetadata. line 259 at base.py: return r[0] There is need to fix documentation.

aicpp avatar May 20 '16 14:05 aicpp

@aicpp thank you for pointing this out, was driving me crazy

garrett5688 avatar May 26 '16 15:05 garrett5688

Has a way to track download progress with files_download_to_file been found/added?

IanButterworth avatar Jan 04 '19 17:01 IanButterworth

@ianshmean No, unfortunately not.

greg-db avatar Jan 04 '19 18:01 greg-db