wordpress-mobile
Unable to upload large videos
Expected behavior
Uploading a very large video (e.g. larger than 500mb) to the Media Library (either from the Media Library itself or via the block editor's Video block) should work without error.
Actual behavior
Sometimes the video fails to upload.
Steps to reproduce the behavior
You'll need a large video for this (over 500mb) for either scenario below.
Scenario 1
- Open the WPiOS app and create a new post using the block editor
- Add a video block
- Choose the large video and wait for it to upload
- Observe the image upload fails (it's hard to say if this will happen every time)
- Go to the app activity log (Profile icon > Help & Support > Activity Logs > Current) and scroll to the bottom to see the latest logs
- Observe an error in the logs similar to:
2020-09-06 20:00:20:394 🔵 Tracked: media_service_upload_failed <error_code: -1001, error_description: Error Domain=NSURLErrorDomain Code=-1001 "The request timed out." UserInfo=
The request timed out., NSErrorFailingURLStringKey=https://public-api.wordpress.com/rest/v1.1/sites/xxxxxxxxxx/media/new?locale=en, NSErrorFailingURLKey=https://public-api.wordpress.com/rest/v1.1/sites/xxxxxxxxxx/media/new?locale=en, _kCFStreamErrorDomainKey=4}, error_domain: NSURLErrorDomain>
Scenario 2
- Open the WPiOS app and navigate to the Media Library
- Add the large video and wait for it to upload
- Observe the image upload fails
- Go to the app activity log (Profile icon > Help & Support > Activity Logs > Current) and scroll to the bottom to see the latest logs
- Observe the same error as before
Reported by user on an iPhone 11 Pro Max, iOS 13.6.1, WPiOS 15.5 (15.5.0.3)
Additional info
- A user who tried uploading a 1.05 gb video could upload the same video using mobile Safari via Gutenberg web (https://wordpress.com/block-editor/) but not from the app (note: WordPress.com recommends videos smaller than 1 GB)
- Cutting the large video in two and uploading each part separately sometimes worked for the user
- The videos in question are .MOV
- Possibly related: https://github.com/wordpress-mobile/gutenberg-mobile/issues/1152
(internal reference: p4a5px-2D2-p2
One possible lead here (noting down for future reference) is that when I background the WPiOS app during a video upload, I can pretty consistently reproduce a failed upload (app needs to be backgrounded for ~30 seconds).
The logs show that the media_service_upload_failed happened as soon as I foregrounded the app (after it was in the background for 30 seconds) and the error is a "The network connection was lost."
2020-10-02 17:39:15:548 🔵 Tracked: editor_video_added <blog_id: 181405924, bytes: 62974446592, duration_secs: 36, ext: mp4, media_origin: not_identified, mime: video/mp4, via: local_library>
2020-10-02 17:39:15:566 🔵 Tracked: media_service_upload_started <blog_id: 181405924>
2020-10-02 17:39:15:600 <WordPress.WordPressAppDelegate: 0x282308990> applicationWillResignActive(_:)
2020-10-02 17:39:15:613 Stats Period: finished persisting Period Stats to disk.
2020-10-02 17:39:16:258 <WordPress.WordPressAppDelegate: 0x282308990> applicationDidEnterBackground(_:)
2020-10-02 17:39:16:268 🔵 Tracked: application_closed <last_visible_screen: Post Editor, time_in_app: 47, time_in_main_reader: 0, time_in_reader_filtered_list: 0, time_in_reader_post: 0>
2020-10-02 17:39:16:276 PingHub disconnecting
2020-10-02 17:39:16:460 PingHub disconnected: WSError(type: Starscream.ErrorType.protocolError, message: "", code: 1000)
2020-10-02 17:39:16:555 TracksService sendQueuedEvents completed. Sent 3 events.
2020-10-02 17:40:02:172 <WordPress.WordPressAppDelegate: 0x282308990> applicationWillEnterForeground(_:)
2020-10-02 17:40:02:172 📜 Resumed encrypted log upload queue due to app entering foreground
2020-10-02 17:40:02:172 PingHub connecting
2020-10-02 17:40:02:214 Error uploading file: The network connection was lost.
2020-10-02 17:40:02:215 🔵 Tracked: media_service_upload_failed <error_code: -1005, error_description: Error Domain=NSURLErrorDomain Code=-1005 "The network connection was lost." UserInfo={_kCFStreamErrorCodeKey=53, NSUnderlyingError=0x281da36c0 {Error Domain=kCFErrorDomainCFNetwork Code=-1005 "(null)" UserInfo={NSErrorPeerAddressKey=<CFData 0x2832cf2a0 [0x1da4a0660]>{length = 16, capacity = 16, bytes = 0x100201bbc0004e160000000000000000}, _kCFStreamErrorCodeKey=53, _kCFStreamErrorDomainKey=1}}, _NSURLErrorFailingURLSessionTaskErrorKey=LocalUploadTask <E93686AA-64D6-478D-986A-4E79F5F6B733>.<198>, _NSURLErrorRelatedURLSessionTaskErrorKey=(
"LocalUploadTask <E93686AA-64D6-478D-986A-4E79F5F6B733>.<198>"
), NSLocalizedDescription=The network connection was lost., NSErrorFailingURLStringKey=https://public-api.wordpress.com/rest/v1.1/sites/181405924/media/new?locale=en, NSErrorFailingURLKey=https://public-api.wordpress.com/rest/v1.1/sites/181405924/media/new?locale=en, _kCFStreamErrorDomainKey=1}, error_domain: NSURLErrorDomain>
2020-10-02 17:40:02:228 Error uploading media: Error Domain=NSURLErrorDomain Code=-1005 "The network connection was lost." UserInfo={_kCFStreamErrorCodeKey=53, NSUnderlyingError=0x281da36c0 {Error Domain=kCFErrorDomainCFNetwork Code=-1005 "(null)" UserInfo={NSErrorPeerAddressKey=<CFData 0x2832cf2a0 [0x1da4a0660]>{length = 16, capacity = 16, bytes = 0x100201bbc0004e160000000000000000}, _kCFStreamErrorCodeKey=53, _kCFStreamErrorDomainKey=1}}, _NSURLErrorFailingURLSessionTaskErrorKey=LocalUploadTask <E93686AA-64D6-478D-986A-4E79F5F6B733>.<198>, _NSURLErrorRelatedURLSessionTaskErrorKey=(
"LocalUploadTask <E93686AA-64D6-478D-986A-4E79F5F6B733>.<198>"
), NSLocalizedDescription=The network connection was lost., NSErrorFailingURLStringKey=https://public-api.wordpress.com/rest/v1.1/sites/181405924/media/new?locale=en, NSErrorFailingURLKey=https://public-api.wordpress.com/rest/v1.1/sites/181405924/media/new?locale=en, _kCFStreamErrorDomainKey=1}
2020-10-02 17:40:02:492 <WordPress.WordPressAppDelegate: 0x282308990> applicationDidBecomeActive(_:)
2020-10-02 17:40:02:494 checkAppleIDCredentialState: No Apple ID found.
2020-10-02 17:40:02:512 🔵 Tracked: application_opened <>
I was not able to diagnose this issue after spending a couple of days on it.
Please note: the sample videos I used were MP4s. I didn't use other formats such as .MOV because I didn't have access to sample in these formats. The videos are:
- Small: https://wordpress.tv/2019/10/08/the-history-of-wordpress-in-four-minutes/ [42 MB]
- Medium: https://wordpress.tv/2019/07/04/matt-mullenweg-matt-on-wordpress/ [385 MB]
- Large: https://wordpress.tv/2018/12/27/state-of-the-word-2018/ [1.25 GB]
Video upload tests
Tests using the WordPress for iOS app
Environment
- WPiOS app logged into a WP.com account with Premium plan (lesser plans restrict uploads)
- Device connected to WiFi: 10 MB/s upload speed (speed varies)
- I did all tests from the My Site → Media screen.
Tests
| Test # | Video | Result | Comment |
|---|---|---|---|
| 1 | Small video | ✅ | Uploaded in 2 minutes. |
| 2 | Medium video | ❌ | Failed after 18 minutes. Note: device had gone to sleep. |
| 3 | Small video | ❌ | † Charles shows "Status: Sending request body…", then "IO: Stream cancelled by CLIENT" |
| 4 | Small video | ❌ | † Charles shows "Status: Failed" and "Failure: EOF" |
| 5 | Small video | ❌ | † Charles shows "Status: Sending request body...", then after a few minutes "Remote server closed the connection before sending response header" |
| 6 | Small video | ⚠️ | Media appeared to fail but after returning to the Media screen, media was in library. Note: Charles proxy was not connected in this test. |
| 7 | Small video | ✅ | Uploaded in 5 minutes. |
† Test performed while monitoring network traffic via Charles proxy.
Observations
It seems to me that uploads failed more often when Charles was used. I got this impression from the above tests and other tests I did.
Another oddity with Charles: even after an upload failed, the request status in Charles was often still be "Sending request body...". This would then change a few minutes after as noted in the above tests.
Tests using WordPress.com on different web browsers
Environment
I also did tests on WordPress.com using different browsers and platforms to see if there were problems there. I didn't use Charles proxy in the below tests.
Tests
| Test # | Video | Result | Comment |
|---|---|---|---|
| 1 | Small video | ✅ | Uploaded via mobile Safari in 5 minutes. |
| 2 | Small video | ✅ | Uploaded via mobile Safari in 5 minutes. |
| 3 | Medium video |
Debugging the WordPress for iOS app via Xcode
Environment
I ran the latest code from the trunk branch and paid special attention to these methods:
uploadMedia:requestEnqueued:success:failure, which calls into:
A timeout error:
See error
2022-02-15 17:01:44:417 WordPress[1319:255550] 🔵 Tracked: media_service_upload_failedA "cannot parse response" error:
See error
2022-02-15 17:31:22:862 WordPress[1397:269654] 🔵 Tracked: media_service_upload_failedThe "cannot parse response" especially deserves more investigation, but I didn't dig into it further.
Next steps
I spent a lot of time with Charles proxy but don't feel like that was the right approach since requests seemed to fail more when the proxy was connected. I think digging further into the issue through Xcode's debugger would be the best next step.
Other options suggested to me are (h/t to @twstokes here):
- Test using a simulator in the cloud (this helps speed up turnaround time on media upload tests which seem to take up to 40 minutes on an internet connection with 10MB/s upload speed.
- Upload media with cURL: Note that the cURL request (copied from Charles) includes the binary blob of media, so it's really large and might be difficult to paste to the terminal
- Play with Network Link Conditioner and use smaller files: Again this might help reduce the turnaround time for media upload tests
Unassigning myself since I wasn't able to find a solution during groundskeeping.
Just wanted to note that this issue may be improved by recent .com server side tweaks. ref: pgbkb-8lq-p2
