iOS upload all photos making app crashing. Files are stuck pending upload

[Luth1ng]

Expected behaviour

The automatic upload function is working as expected except for a specific option : upload all photos from phone.

Actual behaviour

I have 5000+ photos to upload. All is starting fine, the first 1000 photos are successfully uploaded. But as some time, the iOS App crashes.

When I reopen the app, the transfer of the remaining photos resumes but the app crashes after only new few photos uploaded.

After that, each time I reopen the app, the transfer queue is stuck and no further photo is uploaded.

Steps to reproduce

1. Reset settings/cache from the app (or install it for the first time)
2. Enable automatic upload and upload all photos from device
3. See the app crashing after some time (10-15 min / 1000+ photos uploaded)
4. Reopen the app, the upload may resumes
5. See the app crashing again (2 min / few photos uploaded)
6. Reopen the app, upload queue is stuck.
7. Try to force upload via Settings>Transfer and see the app crashes

Logs from the iOS app (from step 3)

2020-11-15 08:56:11 Network request started: PUT https://myurl/remote.php/webdav/Photos/19-07-29%2010-43-31%200611.jpg
2020-11-15 08:56:11 Network request started: PUT https://myurl/remote.php/webdav/Photos/19-07-29%2010-43-23%200610.jpg
2020-11-15 08:56:12 Upload complete https://myurl/remote.php/webdav/Photos/19-07-29 10-43-31 0611.jpg, result: success(4410308 bytes)
2020-11-15 08:56:12 Upload complete https://myurl/remote.php/webdav/Photos/19-07-29 10-43-23 0610.jpg, result: success(4425741 bytes)
2020-11-15 08:56:12 PROCESS-AUTO-UPLOAD find 2 items
2020-11-15 08:56:12 Network request started: PUT https://myurl/remote.php/webdav/Photos/19-07-29%2010-44-41%200612.jpg
2020-11-15 08:56:12 Network request started: PUT https://myurl/remote.php/webdav/Photos/19-07-29%2010-44-48%200613.jpg

**[app crashed]**

2020-11-15 08:57:28 Start session with level 1 Nextcloud Coherence for iOS 3.0.13.1 © 2020
2020-11-15 08:57:28 Request Service Server Nextcloud
2020-11-15 08:57:28 initialize Main
2020-11-15 08:57:28 Application did finish launching
2020-11-15 08:57:28 Network request started: GET https://myurl/ocs/v2.php/cloud/user?format=json
2020-11-15 08:57:28 Network request started: GET https://myurl/status.php
2020-11-15 08:57:28 The application did become active
2020-11-15 08:57:28 Network request started: PROPFIND https://myurl/remote.php/webdav

[...]

2020-11-15 08:57:29 Automatic upload, no new assets found

Screenshots from the app (queue transfer stuck and settings) :

img1 img2

Reasoning or why should it be changed/implemented?

The function simply doesn't work. I have to backup all my photos manually/with a different way than using the app.

iOS version

14.1 iPhone XR

App version

3.0.13.1

Server configuration

Operating system: Debian 10 up to date

Web server: Server version: Apache/2.4.38 (Debian) Server built: 2020-08-25T20:08:29

Database: MariaDB : Server version: 10.3.25-MariaDB-0+deb10u1 Debian 10

PHP version: PHP 7.4.12 (cli) (built: Nov 3 2020 14:31:48) ( NTS )

Nextcloud version: (see Nextcloud admin page) Nextcloud 20.0.1

[bitorius]

Same issue. I'm leaving it at the main screen and it appears okay. But if I click around while uploading, it crashes.

[metroveinte]

+1 Exact the same problem.

[charles-997]

I am also having the same issue.

I am in progress uploading about 18k photos from iPhone Xr. I have been having more success than @Luth1ng however the same problem occurs.

My process has been as follows:

1. Reset Nextcloud cache
2. Flip switch for "Upload the whole camera roll"
3. Nextcloud app will restart from the beginning. (I do not believe already uploaded photos will re-upload and thus duplicate on the NC server- rather NC iOS simply goes through all 18k photos and re-checks for each photo's presence on NC server. Those not present are uploaded.
4. NC app crashes - restart at step 1.

I've been able to get through about 10k of the 18k before crashing. For this initial upload, I have my iPhone plugged into charger, auto-lock disabled, and Nextcloud app open (visible on screen, and phone not being used for anything else)

I think the best solution to this would be to create a bounty that relates to improving the Nextcloud auto upload feature on iOS. I am happy to donate some coffees to this project. I think it would bring a lot of new users to Nextcloud since Google just discontinued Google Photos unlimited support! https://www.bountysource.com/teams/nextcloud

[tedtms]

Seconded. Total photos are around 8k, sync will go until it gets down to about 3.5k and then just halt. No further uploads are made, and the notification badge stays in the 3k range.

[marinofaggiana]

Please don't duplicate the issue !!

[haralddvorak]

I have an additional observation to contribute: In the upload queue most of the files to be uploaded have no thumbnail but a grey icon instead. Those files can be force-uploaded without a problem. I did this for a couple hundred today. A few files though have a thumnail preview instead of the grey icon. Those cannot be force-uploaded. Long-pressing on these files does not show the force-upload button. I'm not sure if this is related but anyway - maybe it helps to resolve the issue.

[marinofaggiana]

I need more info, please use the TestFlight version, so I can look, if possible, where is the issue,

[Luth1ng]

Hello @marinofaggiana

I also tried with the TestFlight version and the behavior is strictly identical.

What do you need exactly ?

[marinofaggiana]

Try with the last version in TestFlight. 3.0.13 b. 4

[changemenemo]

https://github.com/nextcloud/ios/issues/1488 I ve made a new issue because I wzsn'r sure to his one was still active sorry. So yeah still the same I ve upload a screens hot of what transfer screen looks like @marinofaggiana i don't Know why this problem did come back. For me, as show I the screen it's stopped around the middle,I was coming from 13k photos. I was uploading my whole library again because I ve made a new nextcloud instance much more powerful with 20.1 So it's not the instance the problem or the maybe the code in nextcloud server but I highly doubt it

[Luth1ng]

hey @marinofaggiana

The behavior is different with this new version (3.0.14.4) :

• Transfer queue is getting stuck quicker than before (only 10+ photos uploaded)
• I can try force upload for some photos without getting the app crashing but the photo reappears on the queue automatically without any thumbnail. The photo seems to be buggy, I cannot try to reforce the upload because the option isn't displayed.
• Still same logs client side : Automatic upload, no new assets found

[changemenemo]

Not working in beta 5 either I did send you the complete logs also a few days ago on [email protected] did you get them?

[Luth1ng]

Hey/salut @boistordu I am currently trying the TestFlight 3.0.14.5 and the queue is not getting stuck as soon as before...

Still 5000+ photos to upload, stay tuned

[changemenemo]

@Luth1ng well I does stay stucked for me still so I don't know

[changemenemo]

At the same time you are lot using the same server environment as me. Me it s a nextcloud vps from cloudamo so it could also get in the way. Did you get the error of oc_helper.php in your server logs also?

[Luth1ng]

@marinofaggiana

Unfortunately, the app crashes after 1000+ successful uploads and the transfer queue is stuck.

Here are logs from the crash :

2020-11-20 07:23:50 Upload complete https://myurl/remote.php/webdav/Photos/Pellicule/2019/05/IMG_0283.jpg, result: success(4564110 bytes)

**[App crashed]**

2020-11-20 07:27:18 Check if location is enabled: authorizationStatus: 3
2020-11-20 07:27:18 The application Will enter in foreground
2020-11-20 07:27:18 Network request started: PROPFIND https://myurl/remote.php/webdav
2020-11-20 07:27:18 The application did become active
2020-11-20 07:27:19 Automatic upload, no new assets found
2020-11-20 07:27:19 Network request started: GET https://myurl/ocs/v2.php/cloud/user?format=json
2020-11-20 07:27:19 Network request started: GET https://myurl/status.php
2020-11-20 07:27:19 Network response request: https://myurl/remote.php/webdav, result: success(1148 bytes)
2020-11-20 07:27:19 Network request started: PROPFIND https://myurl/remote.php/webdav/
2020-11-20 07:27:19 Network response request: https://myurl/ocs/v2.php/cloud/user?format=json, result: success({
[...]
2020-11-20 07:27:19 Network response request: https://myurl/status.php, result: success({
    edition = "";
    extendedSupport = 0;
    installed = 1;
    maintenance = 0;
    needsDbUpgrade = 0;
    productname = Nextcloud;
    version = "20.0.1.1";
    versionstring = "20.0.1";
})
2020-11-20 07:27:19 Network request started: REPORT https://myurl/remote.php/dav/files/luthing
2020-11-20 07:27:19 Network request started: GET https://myurl/index.php/avatar/luthing/128
2020-11-20 07:27:19 Network request started: GET https://myurl/ocs/v1.php/cloud/capabilities?format=json
2020-11-20 07:27:19 Network response request: https://myurl/remote.php/webdav/, result: success(3142 bytes)
2020-11-20 07:27:19 Network response request: https://myurl/remote.php/dav/files/luthing, result: success(156 bytes)
2020-11-20 07:27:20 Network request started: GET https://myurl/ocs/v2.php/apps/files_sharing/api/v1/shares?reshares=false&subfiles=false
2020-11-20 07:27:20 Network request started: GET https://myurl/ocs/v2.php/apps/files/api/v1/directEditing?format=json
2020-11-20 07:27:20 Network response request: https://myurl/ocs/v2.php/apps/files/api/v1/directEditing?format=json, result: success({
[...]
})
2020-11-20 07:27:20 Network response request: https://myurl/ocs/v2.php/apps/files_sharing/api/v1/shares?reshares=false&subfiles=false, result: success(2404 bytes)
2020-11-20 07:27:55 Check if location is enabled: authorizationStatus: 3
2020-11-20 07:27:55 The application Will enter in foreground
2020-11-20 07:27:55 Network request started: PROPFIND https://myurl/remote.php/webdav
2020-11-20 07:27:55 The application did become active
2020-11-20 07:27:56 Network response request: https://myurl/remote.php/webdav, result: success(1148 bytes)
2020-11-20 07:27:56 Network request started: PROPFIND https://myurl/remote.php/webdav/
2020-11-20 07:27:56 Network response request: https://myurl/remote.php/webdav/, result: success(3142 bytes)
2020-11-20 07:27:56 Automatic upload, no new assets found
2020-11-20 07:27:56 Network request started: GET https://myurl/ocs/v2.php/cloud/user?format=json
2020-11-20 07:27:56 Network request started: GET https://myurl/status.php
2020-11-20 07:27:57 Network response request: https://myurl/status.php, result: success({
    edition = "";
    extendedSupport = 0;
    installed = 1;
    maintenance = 0;
    needsDbUpgrade = 0;
    productname = Nextcloud;
    version = "20.0.1.1";
    versionstring = "20.0.1";
})
2020-11-20 07:27:57 Network response request: https://myurl/ocs/v2.php/cloud/user?format=json, result: success({
[...]
})
2020-11-20 07:27:57 Network request started: REPORT https://myurl/remote.php/dav/files/luthing
2020-11-20 07:27:57 Network request started: GET https://myurl/index.php/avatar/luthing/128
2020-11-20 07:27:57 Network request started: GET https://myurl/ocs/v1.php/cloud/capabilities?format=json
2020-11-20 07:27:57 Network response request: https://myurl/remote.php/dav/files/luthing, result: success(156 bytes)
2020-11-20 07:27:57 Network request started: GET https://myurl/ocs/v2.php/apps/files_sharing/api/v1/shares?reshares=false&subfiles=false
2020-11-20 07:27:57 Network request started: GET https://myurl/ocs/v2.php/apps/files/api/v1/directEditing?format=json
2020-11-20 07:27:57 Network response request: https://myurl/ocs/v2.php/apps/files/api/v1/directEditing?format=json, result: success({[...]
})
2020-11-20 07:27:57 Network response request: https://myurl/ocs/v2.php/apps/files_sharing/api/v1/shares?reshares=false&subfiles=false, result: success(2404 bytes)

**[Nothing happens]**

yes @boistordu I'm aslo having the error server logs side :

{"reqId":"9rbbWNqKKWTurYsf0OHg","level":3,"time":"2020-11-20T06:23:31+00:00","remoteAddr":"<clientIP>","user":"myuser","app":"PHP","method":"PROPFIND","url":"/remote.php/webdav/","message":{"Exception":"Error","Message":"Undefined offset: 3 at /my/path/nextcloud/lib/private/legacy/OC_Helper.php#548","Code":0,"Trace":[{"file":"/my/path/nextcloud/lib/private/legacy/OC_Helper.php","line":548,"function":"onError","class":"OC\\Log\\ErrorHandler","type":"::"},{"file":"/my/path/nextcloud/apps/dav/lib/Connector/Sabre/Directory.php","line":331,"function":"getStorageInfo","class":"OC_Helper","type":"::"},{"file":"/my/path/nextcloud/3rdparty/sabre/dav/lib/DAV/CorePlugin.php","line":799,"function":"getQuotaInfo","class":"OCA\\DAV\\Connector\\Sabre\\Directory","type":"->"},{"file":"/my/path/nextcloud/3rdparty/sabre/dav/lib/DAV/PropFind.php","line":96,"function":"Sabre\\DAV\\{closure}","class":"Sabre\\DAV\\CorePlugin","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/my/path/nextcloud/3rdparty/sabre/dav/lib/DAV/CorePlugin.php","line":802,"function":"handle","class":"Sabre\\DAV\\PropFind","type":"->"},{"file":"/my/path/nextcloud/3rdparty/sabre/event/lib/WildcardEmitterTrait.php","line":89,"function":"propFind","class":"Sabre\\DAV\\CorePlugin","type":"->"},{"file":"/my/path/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":1063,"function":"emit","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/my/path/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":989,"function":"getPropertiesByNode","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/my/path/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":1678,"function":"getPropertiesIteratorForPath","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/my/path/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":1661,"function":"writeMultiStatus","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/my/path/nextcloud/3rdparty/sabre/dav/lib/DAV/CorePlugin.php","line":363,"function":"generateMultiStatus","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/my/path/nextcloud/3rdparty/sabre/event/lib/WildcardEmitterTrait.php","line":89,"function":"httpPropFind","class":"Sabre\\DAV\\CorePlugin","type":"->"},{"file":"/my/path/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":474,"function":"emit","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/my/path/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":251,"function":"invokeMethod","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/my/path/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":319,"function":"start","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/my/path/nextcloud/apps/dav/appinfo/v1/webdav.php","line":84,"function":"exec","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/my/path/nextcloud/remote.php","line":167,"args":["/my/path/nextcloud/apps/dav/appinfo/v1/webdav.php"],"function":"require_once"}],"File":"/my/path/nextcloud/lib/private/Log/ErrorHandler.php","Line":91,"CustomMessage":"--"},"userAgent":"Mozilla/5.0 (iOS) Nextcloud-iOS/3.0.14","version":"20.0.1.1"}

**[app crashed]**

{"reqId":"9OSVic863AlOTZ1T9Sxn","level":3,"time":"2020-11-20T06:27:19+00:00","remoteAddr":"<clientIP>","user":"myuser","app":"PHP","method":"PROPFIND","url":"/remote.php/webdav","message":{"Exception":"Error","Message":"Undefined offset: 3 at /my/path/nextcloud/lib/private/legacy/OC_Helper.php#548","Code":0,"Trace":[{"file":"/my/path/nextcloud/lib/private/legacy/OC_Helper.php","line":548,"function":"onError","class":"OC\\Log\\ErrorHandler","type":"::"},{"file":"/my/path/nextcloud/apps/dav/lib/Connector/Sabre/Directory.php","line":331,"function":"getStorageInfo","class":"OC_Helper","type":"::"},{"file":"/my/path/nextcloud/3rdparty/sabre/dav/lib/DAV/CorePlugin.php","line":799,"function":"getQuotaInfo","class":"OCA\\DAV\\Connector\\Sabre\\Directory","type":"->"},{"file":"/my/path/nextcloud/3rdparty/sabre/dav/lib/DAV/PropFind.php","line":96,"function":"Sabre\\DAV\\{closure}","class":"Sabre\\DAV\\CorePlugin","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/my/path/nextcloud/3rdparty/sabre/dav/lib/DAV/CorePlugin.php","line":802,"function":"handle","class":"Sabre\\DAV\\PropFind","type":"->"},{"file":"/my/path/nextcloud/3rdparty/sabre/event/lib/WildcardEmitterTrait.php","line":89,"function":"propFind","class":"Sabre\\DAV\\CorePlugin","type":"->"},{"file":"/my/path/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":1063,"function":"emit","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/my/path/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":989,"function":"getPropertiesByNode","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/my/path/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":1678,"function":"getPropertiesIteratorForPath","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/my/path/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":1661,"function":"writeMultiStatus","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/my/path/nextcloud/3rdparty/sabre/dav/lib/DAV/CorePlugin.php","line":363,"function":"generateMultiStatus","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/my/path/nextcloud/3rdparty/sabre/event/lib/WildcardEmitterTrait.php","line":89,"function":"httpPropFind","class":"Sabre\\DAV\\CorePlugin","type":"->"},{"file":"/my/path/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":474,"function":"emit","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/my/path/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":251,"function":"invokeMethod","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/my/path/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":319,"function":"start","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/my/path/nextcloud/apps/dav/appinfo/v1/webdav.php","line":84,"function":"exec","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/my/path/nextcloud/remote.php","line":167,"args":["/my/path/nextcloud/apps/dav/appinfo/v1/webdav.php"],"function":"require_once"}],"File":"/my/path/nextcloud/lib/private/Log/ErrorHandler.php","Line":91,"CustomMessage":"--"},"userAgent":"Mozilla/5.0 (iOS) Nextcloud-iOS/3.0.14","version":"20.0.1.1"}

[marinofaggiana]

I don't have crash register for the 3.0.14 ...

[changemenemo]

@marinofaggiana okey no more crashes with the beta 5 indeed and no more stucked. but I deactivated the auto upload in te background so I don t know if in the background it crashes or not. I had been able to upload in a few hours all my library but I had to start from scratch again. So you might look into it. Maybe the crashes corrupts the archive or something.

I have still 2 files who are stucked with 0 bytes. And nextcloud isn't releasing the 8G of cache even so that I have deactivate the cache a long time ago.

[Luth1ng]

I don't have crash register for the 3.0.14 ...

I am currenty giving another try. 1500+ photos uploaded by now, stay tuned!

[changemenemo]

I don't have crash register for the 3.0.14 ...

What are you doing with my mails? Are they going to spam? Can't we have another mail address than your [email protected]? Because it's getting upsetting that any of our mails isn't going through. I ve sent your à dozens of mails since what? Beginning of summer? And you always told me that you received nothing

[marinofaggiana]

@boistordu email ? I receive every day 3, 400 email I don't have time ... please use only GitHub ...

[changemenemo]

@boistordu email ? I receive every day 3, 400 email I don't have time ... please use only GitHub ...

Well I wasn't going to send a complete log in cleartext over github was I? So yes I had crashes with beta 1 and 2 of that same version.

[changemenemo]

2020-11-20 10:42:55 PROCESS-AUTO-UPLOAD find 2 items 2020-11-20 10:42:55 Network start upload file: redacted/2019/03/19-03-21 21-38-10 0464.jpg 2020-11-20 10:42:55 Network start upload file: redacted/Trashbin2/2019/04/19-04-27 15-58-53 0637.jpg 2020-11-20 10:42:55 Upload error 0 length redacted/2019/04/19-04-27 15-58-53 0637.jpg, result: success(0 bytes) 2020-11-20 10:42:55 Network completed upload file: redacted/2019/04/19-04-27 15-58-53 0637.jpg 2020-11-20 10:42:55 Upload error 0 length redacted/2019/03/19-03-21 21-38-10 0464.jpg, result: success(0 bytes)

[changemenemo]

Cache got released after a reboot so that's Okey.

[Luth1ng]

@marinofaggiana

Well, the app crashed after 3500+ photo uploads... The transfer queue gets stuck again...

Stilll the same logs :

2020-11-20 12:21:18 Network request started: PUT https://myurl/remote.php/webdav/Photos/Pellicule/2020/03/IMG_2489.JPG
2020-11-20 12:21:19 Upload complete https://myurl/remote.php/webdav/Photos/Pellicule/2020/03/IMG_2489.JPG, result: success(1551483 bytes)
2020-11-20 12:21:19 PROCESS-AUTO-UPLOAD find 1 items
2020-11-20 12:21:21 Start session with level 1 Nextcloud Coherence for iOS 3.0.14.5 © 2020 (Simulator / TestFlight)
2020-11-20 12:21:21 Check if location is enabled: authorizationStatus: 3
2020-11-20 12:21:21 Request Service Server Nextcloud
2020-11-20 12:21:21 initialize Main
2020-11-20 12:21:21 Application did finish launching
2020-11-20 12:21:22 Automatic upload, no new assets found

[johkoenig]

Some issues with stuck uploads and crashing app. Possible duplicates: #1477 #1451 #1445 #1360 #1275 #1211 #1018 #967 #770

[StefanCohen]

I'm experiencing this problem on one out of three devices (different user accounts on the same server).

iPhone Xr iOS14.2: working iPhone 8 iOS 13.5.1: Working iPhone 6s iOS 12.4.1: not working

I'm upgrading iOS on the 6s to 14.2 to see if that resolves the issue.

[ne20002]

With latest version I now get uploads, but always one upload and another upload right afterwards stating file has been changed. We have this some time ago and it has not been occured since 3.0. Now it is back. Also uploads now stuck and only opening the app starts a new upload sequence of up to five files plus duplicated uploads. Then it stucks again.

[githubnewbie1]

I'm experiencing this problem on one out of three devices (different user accounts on the same server).

iPhone Xr iOS14.2: working iPhone 8 iOS 13.5.1: Working iPhone 6s iOS 12.4.1: not working

I'm upgrading iOS on the 6s to 14.2 to see if that resolves the issue.

iOS 14.3 same issue.

[githubnewbie1]

iOS 14.3 Docker 3.0.4 NC 20.0.4

I have a standard Docker config running on SQLite.

15K odd photos hangs at 11K mark.