droxi
droxi copied to clipboard
jangler
Add timeout and retry during uploading
Hi there,
Once again thanks for this great project I've been using to upload my server backups to dropbox. Lately I have noticed something though, as the upload fails sometimes. After checking it out I find droxi process keeps hanging for days. Maybe this is a sign that your upload code does not handle timeout and retry as resilient as it should. Can you do something about this?
If you need any log or additional info, please tell me.
Best regards,
As I understand it, the problem was probably introduced by my solution to issue #2, which was to start using Dropbox's ChunkedUploader API and automatically resume when the API returns an error--which is what you want when uploading a large file and you briefly lose your net connection halfway through. The problem is that when running non-interactively and something goes wrong that can't be fixed with a retry--like the connection "going bad" or something like that--the program will just loop indefinitely in that state.
One thing the program could do is to try to decipher the type of error that the API is returning and act accordingly, but that seems error-prone. Another solution could be to have certain number of retries before giving up on the upload (possibly specified as a put flag or as a value in the rc file).
Having the application hang indefinitely is certainly not good if it is to be a robust option for scripting. I'll look into a fix this week. Let me know if you have any feedback on my thoughts above.
Yes, simply retrying a few times is easier but I guess the correct way to handle it is to handle the error message accordingly as long as the API error message is consistent. The latter is hard to say without prior knowledge about the API and I'm sure you have more experience with this than me.
For me, after finding out that droxi can hang indefinitely like this I've been using it with timeout like this: timeout 15m droxi put -f -O ${RM_DIR} ${BACKUP_DIR}/*
So all I wish is a retry mechanism, for each file in a put command if possible and a quiet option as I use this in a script anyway and having it print out progress just produces some garbled data in journald.
Best regards,
Wow! I didn't even think about the progress going to a file. I'll definitely add a -q option as well.
Please don't forget to add the retry count on put per file not on command. For example with my command above: droxi put -f -O ${RM_DIR} ${BACKUP_DIR}/* It will put all file under ${BACKUP_DIR} to ${RM_DIR} so please make the retry count reset per file if possible.
Working on a fix, but I'm no longer convinced that the problem is in my code. If I turn off my network interface while doing a chunked upload, it doesn't look like the Dropbox API upload call ever returns.
Committed the changes for the -q and -t options, which are in the new version 0.2.3. By default the chunked upload will retry a few times in case of Dropbox API errors, then abort. I'm guessing this will not fix your problem. The next step would be to call the Dropbox API's upload on another thread, and kill it if it takes too long.
Just updated my build. Will let it run for a few days and tell you the result. If it's like you said though, I doubt that it will fix my issue but yeah, let's try it first.
Yep, you're right. This does not fix my problem, just having it hang there for 8 hours. It's likely you need to do what you said after all.
OK. I will test to see if other operations also have the potential to hang, and wrap them in timeouts.
If possible please count timeout like any dropbox api failure and retry with -t, that should fix my problem for good with resorting to dirty shell scripting.