[FEATURE] Support backup photos and videos that were uploaded on iCloud

[Sixth2538]

Describe the bug When backing up 30,000 photo and video assets from iPhone, uploading is stuck at after 3,000 uploads. Upload status will also show progress of uploading, but number of "Backup" photo does not increase.

Optimise iPhone storage option is selected on this device.

However, newest photos will get uploaded as normal. Once the new photos finish uploading, uploading progress will continue but older photo will not get backed up.

Task List

Please complete the task list below. We need this information to help us reproduce the bug or point out problems in your setup. You are not providing enough info may delay our effort to help you.

• [Y] I have read thoroughly the README setup and installation instructions.
• [Y] I have included my docker-compose file.
• [Y] I have included my redacted .env file.
• [Y] I have included information on my machine, and environment.

Docker Compose:

version: "3.8"

services: immich-server: image: altran1502/immich-server:release entrypoint: ["/bin/sh", "./start-server.sh"] volumes: - ${UPLOAD_LOCATION}:/usr/src/app/upload env_file: - .env environment: - NODE_ENV=production depends_on: - redis - database restart: always networks: - traefik-public

immich-microservices: image: altran1502/immich-server:release entrypoint: ["/bin/sh", "./start-microservices.sh"] volumes: - ${UPLOAD_LOCATION}:/usr/src/app/upload env_file: - .env environment: - NODE_ENV=production depends_on: - redis - database restart: always networks: - traefik-public

immich-machine-learning: image: altran1502/immich-machine-learning:release entrypoint: ["/bin/sh", "./entrypoint.sh"] volumes: - ${UPLOAD_LOCATION}:/usr/src/app/upload env_file: - .env environment: - NODE_ENV=production depends_on: - database restart: always networks: - traefik-public

immich-web: image: altran1502/immich-web:release entrypoint: ["/bin/sh", "./entrypoint.sh"] env_file: - .env restart: always networks: - traefik-public

redis: container_name: immich_redis image: redis:6.2 restart: always networks: - traefik-public

database: container_name: immich_postgres image: postgres:14 env_file: - .env environment: POSTGRES_PASSWORD: ${DB_PASSWORD} POSTGRES_USER: ${DB_USERNAME} POSTGRES_DB: ${DB_DATABASE_NAME} PG_DATA: /var/lib/postgresql/data volumes: - [local bind volume]:/var/lib/postgresql/data restart: always networks: - traefik-public

immich-proxy: container_name: immich_proxy image: altran1502/immich-proxy:release ports: - 2283:80 logging: driver: none depends_on: - immich-server labels: - traefik.enable=true - traefik.docker.network=traefik-public - traefik.http.routers.photoview.rule=Host(photo.xxx.xxx) - traefik.http.routers.photoview.entrypoints=https - traefik.http.routers.photoview.tls=true - traefik.http.routers.photoview.tls.certresolver=le restart: always networks: - traefik-public

networks: traefik-public: external: true

Environmental Variables:

DB_HOSTNAME=immich_postgres DB_USERNAME=xxx DB_PASSWORD=xxx DB_DATABASE_NAME=immich REDIS_HOSTNAME=immich_redis UPLOAD_LOCATION=[local bind volume] JWT_SECRET=xxx ENABLE_MAPBOX=true MAPBOX_KEY=xxx VITE_SERVER_ENDPOINT=https://photo.xxx.xxx/api

To Reproduce

1. Open Immich app on iOS
2. Click back up
3. Select "Recent (All)" album
4. Start backup

Expected behavior Upload all 30,000 assets to server. Continue uploading older photo as shown on progress bar.

Screenshots

System

• Phone OS [iOS, Android]: 15.6
• Server Version: 1.22.0
• Mobile App Version: 1.22.0 build 42

Additional context Add any other context about the problem here.

[MalteKiefer]

I think you must disable the Optimize system. The images must be on the device

[20pins]

I think you must disable the Optimize system.

The images must be on the device

I wonder how many other iOS issues are a result of this. I also have Optimize enabled. I currently have 3570 of 13k items uploaded.

[alextran1502]

If the assets are on iCloud, they won't be uploaded at this stage. I will look into this issue when I implemented Live Photo feature

[Sixth2538]

Thank you for the info and rapid development of the app.

Seems like choosing optimizing storage is the root of the problem. Looking forward to see if this can be sorted out. Seems like it is by design to tie people down with the system.

I have tried various method to download my entire library and there really is no easy way to do it. If this feature can be implemented, it could be a game changer for many iOS users.

[esmondmissen]

Oddly enough I have optimise storage on and the backup is working fine. I've noticed it's a lot slower, but still works for me. I am on iOS 16 however.

[MalteKiefer]

How many photos do you have?

[esmondmissen]

About 17k, I definitely don’t have enough storage on my device to have them all locally.

[2600box]

I am on 15.1 and no photos from my Photos.app show up in Immich. I have the icloud optimised storage turned on.

Interesting if iOS16 is in fact the determining factor...

[Sixth2538]

Happy to report back that iOS 16 backup function is working and backing up all photo and video assets slowly. Even with optimised storage turned on.

[alextran1502]

@Sixth2538 WOOO! This is good news! I guess we can reduce additional code thanks to iOS 16 :)

[gorootde]

Any updates on this one? I am on 16.1.1 and still no photos show up at all (optimize storage enabled)

[jwinut]

I just started using immich. great product!

I also encounter this problem from iCloud Shared Album. I am subscribing to the albums and don't have those images on my iPhone (images not taken by me and I disable iCloud Photos on my device/ Shared Album enable).

I notice that the upload stuck at 100% on the same file name but when I Cancel the backup and start again, immich app backup 3-5 images before stuck at 100% again.

I kept smashing Cancel and Start Backup until my Shared Album is backed up (~100 images + some video).

my device: iPhone X iOS 15.7.1

[alextran1502]

@jwinut There are some differences in the internal APIs of iOS in term of accessing the file system between version 15 and 16 I believe. Would it be possible for your phone to update to iOS 16?

[jwinut]

@alextran1502 it is possible. Would it help you with your development? if yes then I will.

[jwinut]

Updated to iOS 16.1.1. same behavior. need to cancel and start backup multiple times to complete remaining backup.