Auto upload chaos when returning to wifi connection

[Byter3]

⚠️ Before posting ⚠️

• [X] This is a bug, not a question or an enhancement.
• [X] I've searched for similar issues and didn't find a duplicate.
• [X] I've written a clear and descriptive title for this issue, not just "Bug" or "Crash".
• [X] I agree to follow Nextcloud's Code of Conduct.

Steps to reproduce

Use the auto upload feature.

Expected behaviour

Upload everything what configured for auto upload, while the app not freezing crashing.

Actual behaviour

Hi, I'm sorry, I'm gonna be thin, with this description, please if itás not helpful delete it, I can make this comment on Google Play as well. So my phone updated to the new client. Since then, when I arrive home under my Wi-Fi. The Nextcloud client just start freezing up, and seemingly crashing or closing. I even got this error under the auto upload queue, that the app has closed. When it finaly starts to upload again. I can't track which file getting uploaded. It was soo chatotic now I have a bunch of duplicate issue to solve, since it seems even the client can't figure out what is already uploaded and what not.

Also it cosumes while only in the background, on cellular connection, so it not upload anything, 20-30% of my battery. (I'm gonna open up another ticket for that.)

Lastly, I'm not saying it even caused my server to crash, but at least it suspicious, that while my client was uploading a high volume of photos and videos my Nextcloud server crashed, which runned for more than two years without any hickups, 6GB RAM 8core VM. Now parts of my Nextcloud is corrupted. (I opened several topics for that, I just wanted to flag it here as well.)

Android version

12

Device brand and model

Galaxy S21 Ultra

Stock or custom OS?

Stock

Nextcloud android app version

3.22.1

Nextcloud server version

24

Using a reverse proxy?

Yes

Android logs

No response

Server error logs

No response

Additional information

No response

[PVince81]

In my setup I have observed that occasionally the Android files client will reupload files from instant upload after they were moved away. In my case I have a custom Python script that moves away the files from the instant upload folder to another location. In this scenario I'd expect the client to not reattempt uploading

I was told that this issue here might be related so I'll put my findings here. Note: I only picked one example file but it happens quite often, not for every file though. It could be a race condition of sorts.

From access log, the file in question is 20221025_200937.jpg "umstiegstation" is my family's instant upload folder.

First instant upload:

Client sees that the file is not there with 404 on HEAD, then uploads with chunking mode, moves the final file, then checks that the file is indeed there.

x.x.x.x - tingting [25/Oct/2022:20:12:10 +0200] "HEAD /remote.php/dav/files/tingting//petry/pics/umstiegstation/20221025_200937.jpg HTTP/1.1" 404 1021 "-" "Mozilla/5.0 (Android) Nextcloud-android/3.22.2"
x.x.x.x - tingting [25/Oct/2022:20:12:12 +0200] "PUT /remote.php/dav/uploads/tingting/c9cf3bcc5616f694868971821876528c/0000000000000000-0000000002307697 HTTP/1.1" 201 603 "-" "Mozilla/5.0 (Android) Nextcloud-android/3.22.2"
x.x.x.x - tingting [25/Oct/2022:20:12:13 +0200] "MOVE /remote.php/dav/uploads/tingting/c9cf3bcc5616f694868971821876528c/.file HTTP/1.1" 201 767 "-" "Mozilla/5.0 (Android) Nextcloud-android/3.22.2"
x.x.x.x - tingting [25/Oct/2022:20:12:14 +0200] "PROPFIND /remote.php/dav/files/tingting//petry/pics/umstiegstation/20221025_200937.jpg HTTP/1.1" 207 2005 "-" "Mozilla/5.0 (Android) Nextcloud-android/3.22.2"
x.x.x.x - tingting [25/Oct/2022:20:12:14 +0200] "HEAD /remote.php/dav/files/tingting//petry/pics/umstiegstation/20221025_200937.jpg HTTP/1.1" 200 916 "-" "Mozilla/5.0 (Android) Nextcloud-android/3.22.2"

Image service works

My image service runs and moves the file away to another folder. Note: the service is operating on received shares (for isolation purposes) so the folder "input" matches "umstiegstation" here. It actually downloads the file, auto rotates and reuploads elsewhere. In the end what matters most for this issue is that the file gets deleted from "umstiegstation".

x.x.x.x - imageservice [25/Oct/2022:21:15:20 +0200] "GET /remote.php/webdav/input/20221025_200937.jpg HTTP/1.1" 200 2320005 "-" "python-requests/2.27.1"
x.x.x.x - imageservice [25/Oct/2022:21:15:21 +0200] "MKCOL /remote.php/webdav/output/2022/2022-10-25/ HTTP/1.1" 405 918 "-" "python-requests/2.27.1"
x.x.x.x - imageservice [25/Oct/2022:21:15:21 +0200] "PUT /remote.php/webdav/output/2022/2022-10-25/20221025_200937.jpg HTTP/1.1" 201 756 "-" "python-requests/2.27.1"
x.x.x.x - imageservice [25/Oct/2022:21:15:21 +0200] "DELETE /remote.php/webdav/input/20221025_200937.jpg HTTP/1.1" 204 558 "-" " python-requests/2.27.1"
x.x.x.x - imageservice [25/Oct/2022:21:15:22 +0200] "PROPFIND /remote.php/webdav/output/2022/2022-10-25/20221025_200937.jpg HTTP /1.1" 207 1082 "-" "python-requests/2.27.1"

Reupload

Android client sees again that the file is not there in "umstiegstation" (the instant upload folder) and reuploads it.

x.x.x.x - tingting [25/Oct/2022:21:19:08 +0200] "HEAD /remote.php/dav/files/tingting//petry/pics/umstiegstation/20221025_200937.jpg HTTP/1.1" 404 505 "-" "Mozilla/5.0 (Android) Nextcloud-android/3.22.2"
x.x.x.x - tingting [25/Oct/2022:21:23:01 +0200] "HEAD /remote.php/dav/files/tingting//petry/pics/umstiegstation/20221025_200937.jpg HTTP/1.1" 404 1021 "-" "Mozilla/5.0 (Android) Nextcloud-android/3.22.2"
x.x.x.x - tingting [25/Oct/2022:21:23:01 +0200] "MKCOL /remote.php/dav/uploads/tingting/01f5272efab4572aeb8782a2c30feedd HTTP/1.1" 201 603 "-" "Mozilla/5.0 (Android) Nextcloud-android/3.22.2"
x.x.x.x - tingting [25/Oct/2022:21:23:02 +0200] "PROPFIND /remote.php/dav/uploads/tingting/01f5272efab4572aeb8782a2c30feedd HTTP/1.1" 207 2091 "-" "Mozilla/5.0 (Android) Nextcloud-android/3.22.2"
x.x.x.x - tingting [25/Oct/2022:21:23:02 +0200] "PUT /remote.php/dav/uploads/tingting/01f5272efab4572aeb8782a2c30feedd/0000000000000000-0000000002307697 HTTP/1.1" 201 603 "-" "Mozilla/5.0 (Android) Nextcloud-android/3.22.2"
x.x.x.x - tingting [25/Oct/2022:21:23:04 +0200] "MOVE /remote.php/dav/uploads/tingting/01f5272efab4572aeb8782a2c30feedd/.file HTTP/1.1" 201 767 "-" "Mozilla/5.0 (Android) Nextcloud-android/3.22.2"
x.x.x.x - tingting [25/Oct/2022:21:23:04 +0200] "PROPFIND /remote.php/dav/files/tingting//petry/pics/umstiegstation/20221025_200937.jpg HTTP/1.1" 207 2005 "-" "Mozilla/5.0 (Android) Nextcloud-android/3.22.2"
x.x.x.x - tingting [25/Oct/2022:21:23:05 +0200] "HEAD /remote.php/dav/files/tingting//petry/pics/umstiegstation/20221025_200937.jpg HTTP/1.1" 200 916 "-" "Mozilla/5.0 (Android) Nextcloud-android/3.22.2"
x.x.x.x - tingting [25/Oct/2022:21:23:05 +0200] "HEAD /remote.php/dav/files/tingting//petry/pics/umstiegstation/20221025_200937.jpg HTTP/1.1" 200 916 "-" "Mozilla/5.0 (Android) Nextcloud-android/3.22.2"

[user13v31]

I think i Have the same Bug. Had it and then updated to 3.22.3RC but still happening. Here's the crash log (sometimes i get this, sometimes the Handy just suddenly reboots, sometimes get for some reason one fingerprint authorisation request after another)

Cause of error

Exception in thread "Thread-23" java.lang.OutOfMemoryError: Failed to allocate a 48 byte allocation with 763488 free bytes and 745KB until OOM, target footprint 201326592, growth limit 201326592; giving up on allocation because <1% of heap free after GC.
    at java.lang.StringFactory.newStringFromChars(StringFactory.java:112)
    at java.lang.StringBuilder.toString(StringBuilder.java:413)
    at sun.security.x509.AVA.toRFC2253CanonicalString(AVA.java:1018)
    at sun.security.x509.RDN.toRFC2253StringInternal(RDN.java:444)
    at sun.security.x509.RDN.toRFC2253String(RDN.java:424)
    at sun.security.x509.X500Name.getRFC2253CanonicalName(X500Name.java:730)
    at sun.security.x509.X500Name.hashCode(X500Name.java:383)
    at javax.security.auth.x500.X500Principal.hashCode(X500Principal.java:487)
    at java.util.HashMap.hash(HashMap.java:338)
    at java.util.LinkedHashMap.get(LinkedHashMap.java:464)
    at okhttp3.internal.tls.BasicTrustRootIndex.<init>(BasicTrustRootIndex.kt:57)
    at okhttp3.internal.platform.Platform.buildTrustRootIndex(Platform.kt:163)
    at okhttp3.internal.platform.Platform.buildCertificateChainCleaner(Platform.kt:160)
    at okhttp3.internal.platform.Android10Platform.buildCertificateChainCleaner(Android10Platform.kt:82)
    at okhttp3.internal.tls.CertificateChainCleaner$Companion.get(CertificateChainCleaner.kt:42)
    at okhttp3.OkHttpClient$Builder.sslSocketFactory(OkHttpClient.kt:870)
    at com.nextcloud.common.PlainClient$Companion.createDefaultClient(PlainClient.kt:62)
    at com.nextcloud.common.PlainClient$Companion.access$createDefaultClient(PlainClient.kt:47)
    at com.nextcloud.common.PlainClient.<init>(PlainClient.kt:45)
    at com.nextcloud.client.network.ClientFactoryImpl.createPlainClient(ClientFactoryImpl.java:94)
    at com.nextcloud.client.network.ConnectivityServiceImpl.isInternetWalled(ConnectivityServiceImpl.java:79)
    at com.owncloud.android.files.services.FileUploader.retryFailedUploads(FileUploader.java:1072)
    at com.owncloud.android.utils.FilesSyncHelper.lambda$restartJobsIfNeeded$0(FilesSyncHelper.java:240)
    at com.owncloud.android.utils.FilesSyncHelper$$ExternalSyntheticLambda0.run(Unknown Source:10)
    at java.lang.Thread.run(Thread.java:920)

App information

• ID: com.nextcloud.client
• Version: 30220351
• Build flavor: generic

Device information

• Brand: samsung
• Device: starlte
• Model: SM-G960F
• Id: SQ3A.220705.003
• Product: arrow_starlte

Firmware

• SDK: 32
• Release: 12
• Incremental: 14612

[joshtrichards]

@Byter3 Can you share your actual auto-upload configuration? Also, donyou have more than one auto-upload folder active and, if so, what is. the config for each (if not all exactly the same). Ideally you can capture some debug logs during one of these events (either via Logcat as described in the README or in a test installation of the Dev version of the client).

@PVince81 That occuring all the time would be expected behavior based on my understanding of auto-uploads. It's the downside to configuring auto-upload to be kept in their original folder (or moved into app folder for that matter) rather than deleted from phone upon upload success. But if it's happening only sometimes, then either something is wrong or I need to review the code to confirm my understanding (or both).

@user13v31 Can you share your auto-upload configuration entry/entries?

[joshtrichards]

P.S. It's entirely possible some of these matters were addressed by #10926 & #10946. So up-to-date feedback on your original reports would also be helpful from each of you on this thread.

[Byter3]

@joshtrichards Current state of affairs: I have a couple of different auto-upload folders set on my phone. Images pdfs and so on... The phone uploads everything just fine when it's on Wi-Fi. The only strange thing happening right now is that after upload, the app starts to complain, that there is a collision between the server version of an image and on the phone version. Ofccourse, there shouldn't be a collision it just got uploaded. Nevertheless, it complains for every single auto-uploded file.