immich-app
Downloading large album fails in webapp on devices with limited memory
The bug
When I try to download an album from the webapp on either Firefox or Brave browser on Android, the app first displays a progress bar that says "Downloading".
- On Brave this fails already at 10% with a 700MB zip file with an error popup on the top right saying "Unable to download files".
- On Firefox, the progress finishes, and it finally opens the normal "Download this file?" dialog. When clicking OK, at that point it actually starts to save the file to the phone's storage, but Firefox crashes before it can finish.
I think how this works is it's actually streaming the zip into the browser's memory, and only after it's finished, it will start "downloading" the file to the storage, rather than behaving like a normal browser file download where file is streamed to the device's storage from the start. Maybe the device's RAM gets too full and Firefox gets killed. This doesn't appear to be a very scalable way of downloading large files, as mobile apps are often very restricted on RAM usage.
Looking at the source code, it seems to stream the data from the server into a client-side blob and only then opening it as a link to initiate the browser download functionality to save it to the storage.
The OS that Immich Server is running on
Docker
Version of Immich Server
v1.96.0
Version of Immich Mobile App
Web
Platform with the issue
- [ ] Server
- [X] Web
- [ ] Mobile
Your docker-compose.yml content
.
Your .env content
.
Reproduction steps
1. Create album with lots of photos
2. Click download icon from the album view in webapp on device with limited memory
3. App gets killed because of too high memory usage
Additional information
No response
In immich-server container log when the Brave error happens:
[31m ERROR[39m [38;5;3m[StreamableFile] [39m[31mPremature close[39m
Error [ERR_STREAM_PREMATURE_CLOSE]: Premature close
at ServerResponse.onclose (node:internal/streams/end-of-stream:159:30)
at ServerResponse.emit (node:events:530:35)
at emitCloseNT (node:_http_server:1031:10)
at Socket.onServerResponseClose (node:_http_server:278:5)
at Socket.emit (node:events:530:35)
at TCP.<anonymous> (node:net:337:12)
I'm getting the same error when downloading a 4GB album. The problem only occur in Edge on windows. Re-attempting the download in Firefox worked.
[Nest] 7 - 05/27/2024, 11:14:38 PM ERROR [ImmichServer] [StreamableFile] Premature close
Error [ERR_STREAM_PREMATURE_CLOSE]: Premature close
at ServerResponse.onclose (node:internal/streams/end-of-stream:159:30)
at /usr/src/app/node_modules/@opentelemetry/context-async-hooks/build/src/AbstractAsyncHooksContextManager.js:50:55
at AsyncLocalStorage.run (node:async_hooks:346:14)
at AsyncLocalStorageContextManager.with (/usr/src/app/node_modules/@opentelemetry/context-async-hooks/build/src/AsyncLocalStorageContextManager.js:33:40)
at ServerResponse.contextWrapper (/usr/src/app/node_modules/@opentelemetry/context-async-hooks/build/src/AbstractAsyncHooksContextManager.js:50:32)
at ServerResponse.emit (node:events:531:35)
at emitCloseNT (node:_http_server:1020:10)
at Socket.onServerResponseClose (node:_http_server:278:5)
at Socket.emit (node:events:531:35)
at TCP.<anonymous> (node:net:338:12)
I'm getting the same error when downloading a 1GB video.
I tried both iOS app and browser.
The version of iOS app is 1.108.0 build.163, and the error log follows:
I also ran into this issue with a 3GB album with immich v1.112.1
- Downloading the album in web worked, being logged in as the owner of the album
- Downloading via the shared link in web did not work in a reproducible way (same laptop, same firefox browser as before) it always stopped at around 2GB (info: still 27 GiB of RAM free at that time)
- 2 other people were also not able to download the album via the shared link
- If relevant: Download archive size in my user preferences is the default 4 GiB (as reference to #10296 )
[Nest] 18 - 08/19/2024, 9:51:32 AM DEBUG [Api:LoggingInterceptor~5gc3jn7h] POST /api/download/info?key=8iGdYUSfJE4mRDKt8ROpjwuWAvnhTbML7i63QS5U5auDYgjdRlbP9PwHzfDbyu39rZA 201 56.97ms 84.112.125.30
[Nest] 18 - 08/19/2024, 9:51:32 AM VERBOSE [Api:LoggingInterceptor~5gc3jn7h] {"albumId":"55bf818e-f7c3-4ef1-a093-184c7ee45c8e"}
[Nest] 18 - 08/19/2024, 9:51:33 AM DEBUG [Api:LoggingInterceptor~okg6x6xu] POST /api/download/archive?key=8iGdYUSfJE4mRDKt8ROpjwuWAvnhTbML7i63QS5U5auDYgjdRlbP9PwHzfDbyu39rZA 200 59.57ms 84.112.125.30
[Nest] 18 - 08/19/2024, 9:51:33 AM VERBOSE [Api:LoggingInterceptor~okg6x6xu] {"assetIds":["35d257e1-160a-4599-a763-3f30e7678346","dde3a2e4-919e-4c71-8b2d-39ac6882dcd8","db92d7ac-a0fd-4b2b-aea7-d7b6e5847522","520c6c57-b53b-4616-8121-e1c376635536","9a80d16d-ec38-4b06-83e1-2886545db07e","976acdff-45e3-4504-a022-e5db53d329b5","0c815f75-1b20-4445-8d82-6587c447a974","c2539c1f-e2e1-41ce-aaf8-3dd84711e304","af848049-5b21-4956-b97e-26cadf8cda2d","5c6c25d3-a21a-4372-b438-51cbc28d112d","66b43d41-68fd-4ce6-9749-e4c26375414f","07e17885-776f-4950-89a4-25f7f7cdd571","d0c6366f-e691-42c6-8138-9b07646cb882","4fb4260d-d68d-437b-8630-b27ddd2a0501","583cedbf-cf5e-437e-9786-24abe0f15405","c0bf47cf-3b9b-4754-8a07-dfcd0f39b2b2","b9dc8844-46cd-4313-86ae-e8f5c3f4e7d9","52fbcafd-fe48-4514-a6f1-cf8a9b2a782d","2e603a2d-60ac-4d26-b47d-16cbfd3515fb","7f1e2fd0-dc61-4127-8218-d78fe4f87ef5","56491739-0a84-49fd-ac42-88bedbf0075c","c4a0e2c7-aa93-4de0-9b85-bc8b244b9b1f","8234d2df-f119-4013-8fc6-27df1c5e217f","2fdcc277-9d06-4d78-beec-7fdeea0bf4bd","63469b52-a395-481e-a0d3-9552d101a08e","365b9ae1-dbac-4b2e-adf0-05b28a9ecdb2","e6109e57-ba5a-4e85-aff6-31a6b11a7865","075c1717-60a3-47ff-bed3-084990c4204a","6f90f2c7-d7a3-4aa2-b523-f30a6ce202f3","3a733f33-ac66-4670-8a73-1a05667685b8","d2e8785f-3c43-4321-a363-bfdc6cf16211","f80d4e2b-6ff7-4bd3-b76c-050f6038c015","409ef7ba-9f61-4df5-8ffa-48b331bf7f81","b18f5d0a-95dd-4d16-a293-d0d71178757e","1e8c3980-cb51-438e-993f-94e26f22a272","1a81b3e8-65b2-4dcd-a2b5-96827fe94fde","2c17f99a-5d58-49c6-a2ac-d2847df14ded","ef846b51-bfe7-4bb4-8025-b2af3bca5f67","7996ea1b-5eb3-4c5d-b2e4-6a07784a23ad","98b8a63f-7669-41de-a724-a9f02ff03aba","c6791c09-ca3a-4f4b-b6ab-27842ddbad9d","67987fdb-764d-4de7-bd5a-2bd006c37b0a","26b8a7fc-aa3e-409e-a608-d5aed258c6f6","6f5705e4-40f6-49cc-90cc-9538bedae55d","d4d5b2ba-2398-4c55-9a1a-f72805013242","ecaa63b1-a812-40f3-9bac-1227b9111628","9a150143-70bf-4041-8076-1539784b6e66","bbe602f7-f061-4956-8a23-2eca0a9e5a30","35cd40d9-e116-4127-a2af-117340d43a77","0a7d52e0-71cd-4af7-8c40-0795109b19df","516ca572-2ceb-4541-8b63-3bcd5b51f510","cc4d54f7-709a-489c-8cef-0ca7380c7367","d584a3ed-8786-43e1-9fe3-b694962b8109","d0ea91c0-491b-42f9-a338-94ebfea25e55","8d1de856-fd5b-417c-af2f-ea14f968fb5e","ab676ddd-9521-481b-aad8-b0828f5ebbd5","9421069b-c0d5-4f5c-83fd-1e43f96fe851","3bcc7f48-9c34-407a-90bf-b0cfcec93820","57ae9edd-21a8-4d1c-a750-e8fedc513b07","5701c0d2-d4b5-4b33-beb4-29e015fdaf9c","56a65ad3-1fa3-4258-b382-7dff55bb5d1b","98a02b5c-8611-425c-8687-a2de458ab6d9","37d56049-4802-40f3-9ea6-a2c16eb3de1c","a40788c2-b579-4f68-b211-e839954f3747","540efc46-bb22-459d-acd7-32194d46b489","61a3a2cd-5a25-4d7e-8772-291ad44e31d6","eceac8ce-4e41-4061-9ef9-0b8d86e0f332","58b653d4-66fd-4c86-a1b2-339f89c192b5","a36d4d16-26fb-4a72-8a7a-7b3ee2f2eaa4","cfad7f50-e342-4eed-9299-86d333f6221a","cb7a9cdd-73d0-4932-8d19-68fd693801a8","57bf896c-1133-4333-896e-bea701c47bc4","8714b078-fecb-4633-8e02-68711987e186","28654816-02f3-41ce-bcb4-b45e404165f2","8f9dd194-83be-4d93-8ddc-9e6564bf2dfc","13e4dfe2-c2a8-485b-8074-281b2b4889ae","b7c2373e-67d8-495f-86f9-c8fbdb937def","d0b05b80-47fc-4e3c-b99d-7000a7308fdf","a4ecd94b-5429-40c4-b05c-9134900124c4","5491a2c2-c96c-4cf7-80ec-3a92a08fb2ce","caaf5924-ebd7-47ba-a983-6e46fc1ddc35","d6c526d6-0e8f-4b57-bac5-07524af09c98","0155e66e-6ea0-46c7-9e4a-0a1519e80a8d","174be437-ca2d-4f7a-bb26-0d9f392c1611","e5e68e8b-652e-4dfc-8310-5ba9fe387f4c","025d2dfd-b2c0-4c30-a890-eb99eddd4d29","c41c5955-85cc-4d3d-8cb1-ef4603079db6","bc1bda96-97ce-47d7-8bc3-a9af5b158911","4c4af68f-9486-4043-a2cb-bc3e1a05a3d0","9b2c6798-8c84-4c83-97cd-cd82293cc2be","2c52dda8-e199-442f-8741-0b2131ecbba7","5f057864-8822-4f38-8e7a-33359e48e5c2","6feacb13-a1f0-49e2-8c84-3f231fa284ca","5d1fb31f-041e-4bb6-bf14-d0782006f60e","9fdf163a-d5b3-49ef-b0db-bb0640ceb676","4ce39891-1d08-4b63-bf4f-fa4d85e8bcbd","ef2af366-7820-4086-8981-7cabf2b2f981","a73e9800-17b1-4dfe-abb1-50f0cf66d1e4","ac29de3c-4661-42f4-b5a3-2bb61491fdbc","628de331-70ac-42e4-9a37-fde241f32021","...and 270 more"]}
[Nest] 18 - 08/19/2024, 9:51:35 AM ERROR [Api:StreamableFile] Premature close
Error [ERR_STREAM_PREMATURE_CLOSE]: Premature close
at ServerResponse.onclose (node:internal/streams/end-of-stream:159:30)
at ServerResponse.emit (node:events:531:35)
at emitCloseNT (node:_http_server:1020:10)
at Socket.onServerResponseClose (node:_http_server:278:5)
at Socket.emit (node:events:531:35)
at TCP.<anonymous> (node:net:339:12)
at TCP.callbackTrampoline (node:internal/async_hooks:130:17)
For me it fails on Chrome 128 on a desktop Arch Linux, but works fine with Firefox 129. I've lots of free memory.
Error messages are the same (Premature close). For me it feels like Chrome is restricting the maximum file size for objects landing in some kind of browser local/memory storage.
Immich v1.115.0
My wife had issues download a few pictures, looking at the logs I saw this issue as well. Not sure whether it was the same issue or not... But I logged her out and then back in, then it worked. Feels totally unrelated to the error message I saw in the logs so maybe it's not related at all. But if anyone else sees this try logging out and let me know :shrug:
Any news on This @team?
Really Need the download feature to work to get rid of GPhotos.
Someone opened #14725 which is slightly more general, so I'll close this in favour of that.
@nicomerl generally if there is any news it will be posted on the issue, so no need to ask for updates. If this is important to you, we would welcome a PR.
