notes-android icon indicating copy to clipboard operation
notes-android copied to clipboard

Consistent timeouts when syncing

Open weierophinney opened this issue 5 years ago • 15 comments

When I open the app, I consistently receive network timeouts when syncing notes from my server. This means that I am no longer receiving new notes from the server, and notes created on my device are not synced to the server.

To Reproduce

Steps to reproduce the behavior:

  1. Open the app
  2. Wait for the initial sync to complete
  3. Receive timeout error

OR

  1. Open the app
  2. Click the "+" button to create a new note
  3. Create the note
  4. Save the note
  5. Wait for sync to complete
  6. Receive timeout error

Expected behavior

Successful sync.

Stacktrace

I receive the following stacktrace in the app:

App Version: 2.10.3
App Version Code: 81
Files App Version Code: 30100190

---

OS Version: 4.4.153-perf+(00WW_1_400_SP01)
OS API Level: 28
Device: DDV_sprout
Model (and Product): Nokia 7.2 (Daredevil_00WW)

---

com.nextcloud.android.sso.exceptions.UnknownErrorException: Read timed out
	at com.nextcloud.android.sso.api.AidlNetworkRequest.performNetworkRequestV2(AidlNetworkRequest.java:172)
	at com.nextcloud.android.sso.api.NextcloudAPI.performNetworkRequestV2(NextcloudAPI.java:148)
	at it.niedermann.owncloud.notes.persistence.NotesClient.requestServer(NotesClient.java:163)
	at it.niedermann.owncloud.notes.persistence.NotesClient.getNotes(NotesClient.java:98)
	at it.niedermann.owncloud.notes.persistence.NoteServerSyncHelper$SyncTask.pullRemoteChanges(NoteServerSyncHelper.java:413)
	at it.niedermann.owncloud.notes.persistence.NoteServerSyncHelper$SyncTask.doInBackground(NoteServerSyncHelper.java:320)
	at it.niedermann.owncloud.notes.persistence.NoteServerSyncHelper$SyncTask.doInBackground(NoteServerSyncHelper.java:284)
	at android.os.AsyncTask$2.call(AsyncTask.java:333)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at android.os.AsyncTask$SerialExecutor$1.run(AsyncTask.java:245)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
	at java.lang.Thread.run(Thread.java:764)

Server

  • Nextcloud-Version: 17.0.3
  • Nextcloud Notes-Version (server app): 3.1.5
  • PHP-Version: 7.3.15

I have around 1700 notes on the server.

Smartphone

  • Nextcloud Notes-Version (android app): 2.10.3
  • F-Droid or Play Store: Play Store
  • Device: Nokia 7.2
  • Android-Version: 9 (Pi)

weierophinney avatar Mar 12 '20 21:03 weierophinney

Huh, 1700 Notes... insane 😄

Are you able to grab a logcat? I would love to see the timestamps to check if it is really that long running and therefore closing the connection.

Can you try to delete the storage of the app and perform a full synchronization with a good wifi, so the initial sync completes fast? Next notes should theoretically sync faster because it will only be incremental one not by another.

stefan-niedermann avatar Mar 13 '20 05:03 stefan-niedermann

bump @weierophinney

stefan-niedermann avatar Mar 18 '20 09:03 stefan-niedermann

I tried deleting the storage and cache of the app, and then doing a full synchronization; same result.

Click to expand log

App Version: 2.10.3
App Version Code: 81
Files App Version Code: 30100190

---

OS Version: 4.4.153-perf+(00WW_1_400_SP01)
OS API Level: 28
Device: DDV_sprout
Model (and Product): Nokia 7.2 (Daredevil_00WW)

---

com.nextcloud.android.sso.exceptions.UnknownErrorException: Read timed out
    at com.nextcloud.android.sso.api.AidlNetworkRequest.performNetworkRequestV2(AidlNetworkRequest.java:172)
    at com.nextcloud.android.sso.api.NextcloudAPI.performNetworkRequestV2(NextcloudAPI.java:148)
    at it.niedermann.owncloud.notes.persistence.NotesClient.requestServer(NotesClient.java:163)
    at it.niedermann.owncloud.notes.persistence.NotesClient.getNotes(NotesClient.java:98)
    at it.niedermann.owncloud.notes.persistence.NoteServerSyncHelper$SyncTask.pullRemoteChanges(NoteServerSyncHelper.java:413)
    at it.niedermann.owncloud.notes.persistence.NoteServerSyncHelper$SyncTask.doInBackground(NoteServerSyncHelper.java:320)
    at it.niedermann.owncloud.notes.persistence.NoteServerSyncHelper$SyncTask.doInBackground(NoteServerSyncHelper.java:284)
    at android.os.AsyncTask$2.call(AsyncTask.java:333)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at android.os.AsyncTask$SerialExecutor$1.run(AsyncTask.java:245)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
    at java.lang.Thread.run(Thread.java:764)

Can you provide instructions on how to do a logcat?

weierophinney avatar Mar 18 '20 14:03 weierophinney

Sure, you can grab any free LogCat-app from F-Droid or the Play Store (for example this one).

Then just start recording in the LogCat-app, do what every you want in the Notes app and after you get the timeout, go back to the LogCat-app and stop recording. The logfile might contain private information, so you better should send it via e-mail at [email protected] instead of attaching it here.

If you have further questions, just let me know.

stefan-niedermann avatar Mar 19 '20 18:03 stefan-niedermann

FWIW I have exactly the same error, even after dumping Android app data. QOwnNotes on Mac and Windows still work fine.

App Version: 2.11.3
App Version Code: 2011003
Files App Version Code: 30100190

---

OS Version: 4.14.111-gb7713fd5d6ed-ab5820689(5831595)
OS API Level: 29
Device: flame
Model (and Product): Pixel 4 (flame)

---

com.nextcloud.android.sso.exceptions.UnknownErrorException: Read timed out
	at com.nextcloud.android.sso.api.AidlNetworkRequest.performNetworkRequestV2(AidlNetworkRequest.java:172)
	at com.nextcloud.android.sso.api.NextcloudAPI.performNetworkRequestV2(NextcloudAPI.java:148)
	at it.niedermann.owncloud.notes.persistence.NotesClient.requestServer(NotesClient.java:162)
	at it.niedermann.owncloud.notes.persistence.NotesClient.getNotes(NotesClient.java:98)
	at it.niedermann.owncloud.notes.persistence.NoteServerSyncHelper$SyncTask.pullRemoteChanges(NoteServerSyncHelper.java:451)
	at it.niedermann.owncloud.notes.persistence.NoteServerSyncHelper$SyncTask.doInBackground(NoteServerSyncHelper.java:358)
	at it.niedermann.owncloud.notes.persistence.NoteServerSyncHelper$SyncTask.doInBackground(NoteServerSyncHelper.java:320)
	at android.os.AsyncTask$3.call(AsyncTask.java:378)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at android.os.AsyncTask$SerialExecutor$1.run(AsyncTask.java:289)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
	at java.lang.Thread.run(Thread.java:919)

jdkruzr avatar Mar 26 '20 19:03 jdkruzr

Here also a logcat would be interesting.

How many notes (number of files and complete size in kB / MB) do you have on your clouds?

stefan-niedermann avatar Mar 26 '20 20:03 stefan-niedermann

Hi, I have the same Problem. But I had it with an earlier version also. The only other change was that I updated the next cloud instance to version 18. I have only like 20-30 Notes about 200kb size.

App Version: 2.11.3
App Version Code: 2011003
Files App Version Code: 30110090

---

OS Version: 4.4.111-17263988(G950FXXS8DTC1)
OS API Level: 28
Device: dreamlte
Model (and Product): SM-G950F (dreamltexx)

---

com.nextcloud.android.sso.exceptions.UnknownErrorException: Read timed out
	at com.nextcloud.android.sso.api.AidlNetworkRequest.performNetworkRequestV2(AidlNetworkRequest.java:172)
	at com.nextcloud.android.sso.api.NextcloudAPI.performNetworkRequestV2(NextcloudAPI.java:148)
	at it.niedermann.owncloud.notes.persistence.NotesClient.requestServer(NotesClient.java:162)
	at it.niedermann.owncloud.notes.persistence.NotesClient.getNotes(NotesClient.java:98)
	at it.niedermann.owncloud.notes.persistence.NoteServerSyncHelper$SyncTask.pullRemoteChanges(NoteServerSyncHelper.java:451)
	at it.niedermann.owncloud.notes.persistence.NoteServerSyncHelper$SyncTask.doInBackground(NoteServerSyncHelper.java:358)
	at it.niedermann.owncloud.notes.persistence.NoteServerSyncHelper$SyncTask.doInBackground(NoteServerSyncHelper.java:320)
	at android.os.AsyncTask$2.call(AsyncTask.java:333)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at android.os.AsyncTask$SerialExecutor$1.run(AsyncTask.java:245)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
	at java.lang.Thread.run(Thread.java:764)

Bricki85 avatar Mar 29 '20 19:03 Bricki85

Can you all please check if there is a hint about an access in your server logs?

Interesting would be (if one has the possibility to) if you could remove all notes from your server and add them one by one back to your cloud (syncing after each note) and check whether this depends on a single special note or on the pure amount.

Also try switching between mobile data and wifi so we can get a hint if this is related to the network type.

How is the sync with the normal files app? Does it work flawlessly?

stefan-niedermann avatar Apr 16 '20 18:04 stefan-niedermann

Hi, i'm having this problem now.

Can you all please check if there is a hint about an access in your server logs?

Interesting would be (if one has the possibility to) if you could remove all notes from your server and add them one by one back to your cloud (syncing after each note) and check whether this depends on a single special note or on the pure amount.

I can confirm adding notes in small chunks seems to work and not trigger the timeout.

A chunk of 213 files (~290KiB) fails, but small chunks of 50 (89KiB) or 100 files (~160KiB) work fine.

My total amount of notes is 382 (2.8MB), the whole amount does not sync.

Also try switching between mobile data and wifi so we can get a hint if this is related to the network type.

In my case, the network type didn't make any difference.

How is the sync with the normal files app? Does it work flawlessly?

The normal files app works flawlessly.

danidoni avatar Jul 07 '20 14:07 danidoni

Hi, and I have the same problem also. I admit my device is a bit obscure but still, I was using older versions for long time without any issues. I upgraded my server recently (upgrade over 9 versions:) and with that also my Notes app. Interesting thing is that it worked initially, but at some point it stopped. I even downgraded the app to 2.15.0 (from the current latest) but it's the same. I don't think I have many notes, 56 actually, some of them shared with/from others.

I can try to get some more logs when I have time, because this makes me sad, I'm already used to the app. And as far as I can tell the normal files app works (at least I haven't noticed any issues so far).

Got this: Files App Version Code: 30120190


OS Version: 3.18.31-perf-g46f6132-01558-g873422d(257) OS API Level: 25 Device: Y3 Manufacturer: YOTA Model (and Product): Y3 (Y3)


com.nextcloud.android.sso.exceptions.NextcloudApiNotRespondingException: Nextcloud files app api is not responding. Please report this issue. at com.nextcloud.android.sso.api.AidlNetworkRequest.waitForApi(AidlNetworkRequest.java:147) at com.nextcloud.android.sso.api.AidlNetworkRequest.performAidlNetworkRequestV2(AidlNetworkRequest.java:278) at com.nextcloud.android.sso.api.AidlNetworkRequest.performNetworkRequestV2(AidlNetworkRequest.java:165) at com.nextcloud.android.sso.api.NextcloudAPI.performNetworkRequestV2(NextcloudAPI.java:148) at it.niedermann.owncloud.notes.persistence.SSOClient.requestFilesApp(SSOClient.java:27) at it.niedermann.owncloud.notes.persistence.NotesClient.requestServer(NotesClient.java:167) at it.niedermann.owncloud.notes.persistence.NotesClientV02.putNote(NotesClientV02.java:40) at it.niedermann.owncloud.notes.persistence.NotesClientV02.createNote(NotesClientV02.java:44) at it.niedermann.owncloud.notes.persistence.NoteServerSyncHelper$SyncTask.pushLocalChanges(NoteServerSyncHelper.java:383) at it.niedermann.owncloud.notes.persistence.NoteServerSyncHelper$SyncTask.doInBackground(NoteServerSyncHelper.java:346) at it.niedermann.owncloud.notes.persistence.NoteServerSyncHelper$SyncTask.doInBackground(NoteServerSyncHelper.java:306) at android.os.AsyncTask$2.call(AsyncTask.java:305) at java.util.concurrent.FutureTask.run(FutureTask.java:237) at android.os.AsyncTask$SerialExecutor$1.run(AsyncTask.java:243) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1133) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:607) at java.lang.Thread.run(Thread.java:761)

dees001 avatar Aug 28 '20 14:08 dees001

@dees001 your issue is different from this one. Please try the steps in our FAQ and open a new issue if the steps do not work for you.

stefan-niedermann avatar Aug 28 '20 14:08 stefan-niedermann

Yeah, well. I guess the probably best solution will be to exclude the content property when fetching the Notes list on the very first import. Then a request for each note should be make so we keep the total size of the requests as low as possible (and it has the nice side effect that we can display a progress bar when importing an account).

For following synchronizations this shouldn't matter since we already use pruneBefore to only transfer the notes that actually have been changed.

It will require some work in the ImportAccountActivity and in the MainActivity (for importing further accounts) but should be doable.

stefan-niedermann avatar May 10 '21 17:05 stefan-niedermann

I'm working on an enhancement of the API, so that you (as client developer) can define a maximum amount of notes that should be transferred in one response. This is the last feature that I want to add to the API version 1.2 and then I will release the app version 4.1. Using this should be more comfortable. :-)

korelstar avatar May 10 '21 18:05 korelstar

Same issue here with 200+ files, F-Droid version of the app v3.4.11, copying files in chunks solves it. Hopefully this gets solves and I don't have to do this each time I get the app installed in a device. Thank you for the good work.

zemua avatar Sep 24 '21 17:09 zemua

So, something is happening here, i just commited the first change in this direction (https://github.com/stefan-niedermann/nextcloud-notes/commit/5801f00bfb79b19b02dc8fcb6862ed8a7bb71a1d) to synchronize notes one by one while importing.

This can (and probably will) of course be extended in the future to use the chunk API provided in the Notes server app 3.4.0. The reason i started to implement it this way is to bring the benefit to all users, regardless which version of the server app is installed. The changes will be shipped in Notes Android 3.4.14.

stefan-niedermann avatar Sep 30 '21 10:09 stefan-niedermann