nextcloud
Syncing/refreshing is extremly slow
This issue respects the following points:
- [x] I have read the FAQ
- [x] I have searched for existing issues
- [x] The issue affects the Notes Android app, neither the Notes server app nor the Nextcloud Android app.
Describe the bug
Refreshing/Syncing the Notes app takes way too long (~10 seconds), even if nothing has changed.
I think this means all remote notes are downloaded on every request, at least thats what I got from quickly glancing over the relevant code.
As per the Nextcloud Notes Api docs: https://github.com/nextcloud/notes/blob/main/docs/api/v1.md#get-all-notes-get-notes
PruneBefore should be used to only download notes that have been remotely changed since the last update:
pruneBefore integer, optional All notes without change before of this Unix timestamp are purged from the response, i.e. only the attribute id is included. You should use the Unix timestamp value from the last request's HTTP response header Last-Modified in order to reduce transferred data size.
(or alternatively the ETAG contained in the response)
This is still an issue even if I'm wrong about the cause, it shouldn't take more than a second (accounting for network latency).
Expected behavior
No response
Notes Android version
3.5
Notes server version
4.11
Nextcloud Android version
3.30.6
Nextcloud version
30.0.4
Device
Samsung Galaxy Z Flip 5
Android Version
14
App Store
- [x] Google Play Store
- [ ] F-Droid
- [ ] Huawei App Gallery
Stacktrace
No response
The sync takes round about 300ms on my personal device (~2000 notes).
You can find the implementation here, as you can see pruneBefore and ETags are both considered.
I have ~450 notes and it's running on a reasonably fast server with nextcloud data on a ssd (30ms ping time from my phone).
Here is a screen recording (switched to favorites to not leak any notes): https://github.com/user-attachments/assets/723e5d58-73af-4e92-bf58-40cd999fc1a3
I don't really know much about android development. How do I profile this to see what's taking so long?
Maybe use curl / wget to check the lateny od your server. If it is low, we can rule out this as bottleneck
Attached a debugger to a build of current master, some observations:
Both etag and modified are always "0".
onlyLocalChanges is always false
All 450 Notes are downloaded in full on each refresh.
@stefan-niedermann I fixed this in my PR, did it truly work for you before? If so I might need to check the headers without taking capitalization into account.
My response headers are as follows:
"x-notes-api-versions" -> "0.2, 1.3"
"Server" -> "cloudflare"
"vary" -> "accept-encoding"
"x-frame-options" -> "SAMEORIGIN"
"last-modified" -> "Mon, 30 Dec 2024 20:52:52 GMT"
"Report-To" -> "{"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v4?s=PHwMMeLx0%2BqVoUZEGRUrzzsRvoRpC%2FcTfocbEFGEC7CpXSv5PPGxXoIb%2BvrxPApOSoZJ4ogf2DOnc7lVXVi5Xw5GUymVLdd8BRthOC60iUwwujNjYmsKYC9X%2BiP9q7GmsWK0OYcRc%2FqwbPR%2FZUFaYvRxLA%3D%3D"}],"group":"cf-nel","max_age":604800}"
"referrer-policy" -> "no-referrer"
"server-timing" -> "cfL4;desc="?proto=TCP&rtt=40726&min_rtt=25225&rtt_var=20532&sent=3&recv=6&lost=0&retrans=0&sent_bytes=203&recv_bytes=1238&delivery_rate=53439&cwnd=251&unsent_bytes=0&cid=aca0bfb80b6fae54&ts=731&x=0""
"Content-Type" -> "application/json; charset=utf-8"
"Transfer-Encoding" -> "chunked"
"x-request-id" -> "R9Reyt0ihOHBCvFmJyxb"
"CF-RAY" -> "8fa4ea80ecad7a46-DUS"
"Connection" -> "keep-alive"
"x-permitted-cross-domain-policies" -> "none"
"cf-cache-status" -> "DYNAMIC"
"Date" -> "Mon, 30 Dec 2024 20:52:52 GMT"
"feature-policy" -> "autoplay 'none';camera 'none';fullscreen 'none';geolocation 'none';microphone 'none';payment 'none'"
"content-security-policy" -> "default-src 'none';base-uri 'none';manifest-src 'self';frame-ancestors 'none'"
"Cache-Control" -> "no-cache, no-store, must-revalidate"
"x-content-type-options" -> "nosniff"
"x-xss-protection" -> "1; mode=block"
"NEL" -> "{"success_fraction":0,"report_to":"cf-nel","max_age":604800}"
"x-robots-tag" -> "noindex, nofollow"
"etag" -> "W/"5180431749b31a5741cffd0209f567c7""
"alt-svc" -> "h3=":443"; ma=86400"
Header names are not case sensitive.
From RFC 2616 - "Hypertext Transfer Protocol -- HTTP/1.1", Section 4.2, "Message Headers":
Each header field consists of a name followed by a colon (":") and the field value. Field names are case-insensitive.
The updating RFC 7230 does not list any changes from RFC 2616 at this part.
Thanks for the PR! Hm. Renaming would probably fix this for you and raise the issue for me (yes, sync does work quickly here on my setup) 😆
I don't have time to check the actual comparison in the source code, but I think the solution is to use equalsIgnoreCase() (or whatever is needed here) rather than renaming the expected header names.
@stefan-niedermann Opened a PR upstream as I think it's cleaner to fix it there.
@lu4p Now where https://github.com/nextcloud/Android-SingleSignOn/pull/796 got closed, would you like to take another shot at upstream or should we work around this in the Notes app directly?
Doing it in the Notes app seems to be a relatively low-hanging fruit for a huge UX improvement. For me, synchronizing is very slow, which is not only annoying but also somewhat dangerous. When I am distracted, I sometimes start modifying a note that isn't up-to-date on my device yet, effectively deleting the latest changes.
