joplin
joplin copied to clipboard
iOS,Android: Dropbox sync broken
Operating system
iOS
Joplin version
13.0.1 (dev, ios)
Desktop version info
No response
Current behaviour
- Enable Dropbox sync
- Add a new file
- Sync
- Observe that a "Network request failed" is logged and the item isn't uploaded.
Expected behaviour
Dropbox sync should work.
Logs
No response
Additional details (originally posted on the forum):
I'm able to reproduce this on an iOS simulator. For me, the failing fetch is requesting
/info.json
from Dropbox. If I use therequestToCurl_
debug method to convert the request to a CURL command[^1]. If I then run this from a terminal, the request is successful.The failing request seems to be using React Native's
fetch
(on this line) (rather thanfetchBlob
oruploadBlob
).[^1]: The CURL request is similar to (may have typos):
curl -X GET -H 'Dropbox-API-Arg: {"path": "/info.json"}' -H 'Authorization: Bearer TOKEN_HERE' -H 'Content-Type: application/octet-stream' https://content.dropboxapi.com/2/files/download
Other users are reporting this on older versions of Joplin.
At some point we had this "Network request failed" error because we weren't passing the "content-type" header (even when it wasn't needed), but in that case it looks like it's specified. Or maybe it should be application/json in this particular request?
A Content-Type
of application/json
doesn't work either. With this, however, the error message is more descriptive:
Synchronizer: [Error: GET files/download: Error (400): Error in call to API function "files/download": Bad HTTP "Content-Type" header: "application/json". Expecting one of "text/plain", "text/plain; charset=utf-8", "application/octet-stream", "application/octet-stream; charset=utf-8".]
Using text/plain
again gives the error,
ERROR 17:40:01: Synchronizer: [TypeError: Network request failed]
Replacing shim.fetch
with shim.debugFetch
(which uses XMLHttpRequest
) and using text/plain
for the content type gives
INFO ======================== XHR ERROR
INFO null
INFO -------------------------------------
INFO The network connection was lost.
INFO ======================== XHR ERROR
These all fail when fetching /info.json
.
Here are a few other things that could be tried:
- [ ] Manually checking whether
files/download
works for other files in the Joplin folder. - [ ] Overwriting
info.json
with other content, then trying to fetch again. - [ ] Upgrading React Native (we use v0.71, the latest version is v0.74).
- Upgrading to RN 0.73.8 doesn't seem to help.
- [ ] Running an identical
fetch
within a WebView.- This results in a similar error:
> await fetch( 'https://content.dropboxapi.com/2/files/download' , {"headers":{"Dropbox-API-Arg":"{\"path\":\"/info.json\"}","Authorization":"Bearer TOKEN_HERE","Content-Type":"application/octet-stream"},"method":"GET"} ) [Error] Failed to load resource: The network connection was lost. (download, line 0) [Error] TypeError: Load failed
- (WebView only) Changing
method
fromGET
toPOST
results in a successfulResponse
. If this change is made to the Dropbox API, sync still fails on iOS.- The historical reason for using GET instead of POST is explained in this comment: https://github.com/laurent22/joplin/blob/8e93f0975f1fe045be5717f818e3c2ea21e91618/packages/lib/file-api-driver-dropbox.js#L126-L135
I've also noticed this issue on an Android 14 emulator (I've only tested with the React Native 0.73.8 upgrade branch, however).
I've also noticed this issue on an Android 14 emulator (I've only tested with the React Native 0.73.8 upgrade branch, however).
With Android maybe it's easier to debug - did you try running adb logcat
at the same time to see if there's more info in the low-level error messages?
Also maybe compare the response between running curl on /info.json
and another file. Maybe Dropbox started adding some more headers to the JSON responses that makes the RN network lib fail.
Also maybe compare the response between running curl on /info.json and another file. Maybe Dropbox started adding some more headers to the JSON responses that makes the RN network lib fail.
I've done additional testing on Android.
fetch
seems to work for a resource file. When running the following from the Hermes development tools, the following is successful (prints the content of a note)[^1]:
fetch("https://content.dropboxapi.com/2/files/download", {"headers":{"Dropbox-API-Arg":"{\"path\":\"/faf15418f15e410da0000ac96d8c4c47.md\"}","Authorization":"Bearer TOKEN_HERE","Content-Type":"application/octet-stream"},"method":"GET"})
.then(e => e.text()).then(t => console.log(t)).catch(e => console.error(e))
Edit: If I run the above fetch
twice (rather than just once), the second and subsequent requests fail.
Changing the path to /info.json
is still not successful:
fetch("https://content.dropboxapi.com/2/files/download", {"headers":{"Dropbox-API-Arg":"{\"path\":\"/info.json\"}","Authorization":"Bearer TOKEN_HERE","Content-Type":"application/octet-stream"},"method":"GET"})
.then(e => e.text()).then(t => console.log(t)).catch(e => console.error(e))
Interestingly, sending a POST
request works:
// First, fetch with "method": "POST"
fetch("https://content.dropboxapi.com/2/files/download", {"headers":{"Dropbox-API-Arg":"{\"path\":\"/info.json\"}","Authorization":"Bearer TOKEN_HERE","Content-Type":"application/octet-stream"},"method":"POST"})
.then(e => e.text()).then(t => console.log(t))
// Fetching a second time, this time with "method": "GET" **also works**:
.then(() =>
fetch("https://content.dropboxapi.com/2/files/download", {"headers":{"Dropbox-API-Arg":"{\"path\":\"/info.json\"}","Authorization":"Bearer TOKEN_HERE","Content-Type":"application/octet-stream"},"method":"GET"})
).then(e => e.text()).then(t => console.log(t))
Adding a third .then(() => fetch( ... )).then(e => e.text()).then(t => console.log(t))
results in an Unhandled Promise Rejection (id: 3): TypeError: Network request failed
, after printing two copies of info.json
.
With Android maybe it's easier to debug - did you try running adb logcat at the same time to see if there's more info in the low-level error messages?
When I run adb logcat
, I get a large number of unrelated errors per second:
05-04 09:10:44.720 539 679 E ClipboardService: Denying clipboard access to com.android.chrome, application is not in focus nor is it a system service for user 0
05-04 09:10:44.720 539 679 E JavaBinder: !!! FAILED BINDER TRANSACTION !!! (parcel size = 108)
05-04 09:10:44.720 539 679 E ClipboardService: Denying clipboard access to com.android.chrome, application is not in focus nor is it a system service for user 0
05-04 09:10:44.720 539 679 E JavaBinder: !!! FAILED BINDER TRANSACTION !!! (parcel size = 108)
05-04 09:10:44.721 539 679 E ClipboardService: Denying clipboard access to com.android.chrome, application is not in focus nor is it a system service for user 0
05-04 09:10:44.721 539 679 E JavaBinder: !!! FAILED BINDER TRANSACTION !!! (parcel size = 108)
05-04 09:10:44.721 539 679 E ClipboardService: Denying clipboard access to com.android
If I search for request
in the output (using adb logcat | grep -i 'request' -B 3
), I get output that doesn't seem particularly helpful:
...
--
05-04 09:17:04.231 539 679 E ClipboardService: Denying clipboard access to com.android.chrome, application is not in focus nor is it a system service for user 0
05-04 09:17:04.232 539 653 W PackageManager: Not removing package com.google.android.trichromelibrary hosting lib com.google.android.trichromelibrary version 631209938 used by [VersionedPackage[com.android.chrome/631209938]] for user 0
05-04 09:17:04.232 539 653 W PackageManager: Not removing package com.google.android.trichromelibrary hosting lib com.google.android.trichromelibrary version 631211838 used by [VersionedPackage[com.google.android.webview/631211838]] for user 0
05-04 09:17:04.236 455 9722 D installd: Device /data has 429649920 free; requested 624066560; needed 194416640
--
05-04 09:17:04.264 539 679 E ClipboardService: Denying clipboard access to com.android.chrome, application is not in focus nor is it a system service for user 0
05-04 09:17:04.267 455 9722 D installd: Refusing to clear cached data in reserved space
05-04 09:17:04.267 455 9722 E installd: Failed to free up 624066560 on /data; final free space 429649920: Success
05-04 09:17:04.271 1365 1476 I ExternalStorageServiceImpl: Free cache requested for 194416640 bytes
05-04 09:17:04.272 539 679 E ClipboardService: Denying clipboard access to com.android.chrome, application is not in focus nor is it a system service for user 0
05-04 09:17:04.278 539 679 E ClipboardService: Denying clipboard access to com.android.chrome, application is not in focus nor is it a system service for user 0
05-04 09:17:04.286 539 679 E ClipboardService: Denying clipboard access to com.android.chrome, application is not in focus nor is it a system service for user 0
05-04 09:17:04.286 1365 1476 I ExternalStorageServiceImpl: Free cache requested for 194416640 bytes
--
05-04 09:17:23.771 539 679 E JavaBinder: !!! FAILED BINDER TRANSACTION !!! (parcel size = 108)
05-04 09:17:23.771 539 679 E ClipboardService: Denying clipboard access to com.android.chrome, application is not in focus nor is it a system service for user 0
05-04 09:17:23.774 539 679 E ClipboardService: Denying clipboard access to com.android.chrome, application is not in focus nor is it a system service for user 0
05-04 09:17:23.776 13576 20973 E ReactNativeJS: '09:17:23: Synchronizer:', [TypeError: Network request failed]
[^1]: Note that Hermes uses code preprocessing to support async
and await
, so .then
is used above.
I've raised this issue on the Dropbox forum (Edit: ~~The post was marked as spam~~. Edit 2: It should now be visible).
Syncing is working on Mac, but not on any of my iOS devices.
To add some symptoms that may help in diagnosis, if I completely delete the iOS app, re-connect the Dropbox link, it syncs fine. Loads all the existing data from the Dropbox vault. But then will not perform any updates after that. Adding or deleting an item on a Mac does not sync to the iOS device. Or in the other direction.
I've opened a pull request with a workaround for the sync issue. It seems that if Joplin tries to download the same file multiple times with a GET
request, the second and subsequent requests fail. The pull request works around the issue by, when a download request fails, requesting another file, then re-trying the original request.
It's strange that repeated GET
requests fail on mobile but not on desktop. Maybe this is related to cookies or other information (e.g. a user-agent string) passed with the request?
To add some symptoms that may help in diagnosis, if I completely delete the iOS app, re-connect the Dropbox link, it syncs fine. Loads all the existing data from the Dropbox vault. But then will not perform any updates after that. Adding or deleting an item on a Mac does not sync to the iOS device. Or in the other direction.
For me, the sync errors happen first when requesting info.json
. One possibility is that uploading a file causes Joplin to make multiple requests for this file, which breaks future sync (Joplin might request info.json
at the beginning of a sync?).
An alternate workaround could be to cache the last request to /files/download
. However, I'm unsure how long cached responses would need to be kept and whether this would break downloading changes to files.
I don't have a lot of experience with Dropbox programming (I'm a real time embedded engineer) but could the difference between mobile and desktop be that on desktop computers Dropbox is a daemon and the app writes directly to the local Joplin folder for requests? Versus on mobile, the GET requests have to be issued to the Dropbox servers directly?
I don't have a lot of experience with Dropbox programming (I'm a real time embedded engineer) but could the difference between mobile and desktop be that on desktop computers Dropbox is a daemon and the app writes directly to the local Joplin folder for requests? Versus on mobile, the GET requests have to be issued to the Dropbox servers directly?
No this is not it. There are three reasons for this issue:
-
Dropbox changed something to their API (not really a bug, but that's the most likely reason)
-
React Native networking library is terrible and fails on basic things that work fine everywhere else.
-
React Native error reporting is terrible. HTTP has been around for 33 years and is very well understood, yet RN can't figure out how to display a proper error message and defaults to "Network request failed" for every error, leaving us clueless as to what's happening.
Re-opening — users are still experiencing Dropbox sync issues: https://discourse.joplinapp.org/t/ios-sync-broke-due-to-network-request-failing-with-dropbox/37915/24.
I’m still having the same issues.
I decided not to comment 30 or more days ago when I saw a fix that was yet to be made a part of main, but yea dropbox sync is borked. I only had the issue after restoring a backup of my phone after a conference. TLDR: still borked.