[BUG] Missing pictures from mobile with full backup

[Isti6github]

The bug

I use the fresh install immich and I found that I have many missing pictures, what not uploded. Backup status 100%

The OS that Immich Server is running on

Intel compute stick - Dietpi OS

Version of Immich Server

V1.52.1

Version of Immich Mobile App

V1.52.0 build 90 IOS

Platform with the issue

• [ ] Server
• [ ] Web
• [X] Mobile

Your docker-compose.yml content

Latest

Your .env content

Latest

Reproduction steps

1.install immich
2. Start backup
3. See the photo list
...

Additional information

No response

[Isti6github]

I forget this picture

[alextran1502]

Do you perhaps have some photos on iCloud?

[Isti6github]

Do you perhaps have some photos on iCloud?

No I havent photos on iCloud

[raisinbear]

Hm, this is what it looks like in the mobile interface for me when experiencing #2115 or #1967. Although I haven’t checked the app log for the error you posted, so this might be something completely different. Do the images show up as uploaded once you run the missing thumbnail and metadata extraction jobs in the web ui?

[alextran1502]

Do you have some of the photos on the device that gets uploaded over other means i.e Web or CLI?

[Isti6github]

Do you have some of the photos on the device that gets uploaded over other means i.e Web or CLI?

No, just other user pictures what uplode over mobile app. Now I try manual update over webui (jobs, every missing)

[Isti6github]

Do you have some of the photos on the device that gets uploaded over other means i.e Web or CLI?

No, just other user pictures what uplode over mobile app. Now I try manual update over webui (jobs, every missing)

Jobs running, but looks like as frozen(not progress)

[Isti6github]

I restarted the container after I run missing jobs and I got same pictures, but some missing still. Now I started again the missing jobs and founded 20 (before was 260). I wait to done it

[Isti6github]

I done with that and still have missing pictures

[raisinbear]

Hm it looks like this was at least part of the problem. Now the web interface reports no more missing thumbnails and metadata when you click on it? Fyi you should not need to have to restart the container afterwards. Worst case force quit and restart the mobile app but usually it should update by itself, too. I suppose you do not see those missing images on the web either? Are you seeing anything suspicious in the container logs, e.g. in the docker logs immich_microservices one?

[Isti6github]

Hm it looks like this was at least part of the problem. Now the web interface reports no more missing thumbnails and metadata when you click on it? Fyi you should not need to have to restart the container afterwards. Worst case force quit and restart the mobile app but usually it should update by itself, too. I suppose you do not see those missing images on the web either? Are you seeing anything suspicious in the container logs, e.g. in the docker logs immich_microservices one?

No I dont see on web those pictures. And now no more missing ones when I run jobs. I found it in microservice log:

[raisinbear]

No I dont see on web those pictures. And now no more missing ones when I run jobs. I found it in microservice log:

That signature looks a lot like #2115.

[raisinbear]

As for the still missing images, one more place to check would be the immich_postgres logs. Anything out of the ordinary in there?

[Isti6github]

As for the still missing images, one more place to check would be the immich_postgres logs. Anything out of the ordinary in there?

I attached it, if you need more just ask and I uplode it. LOG_immich_postgres.txt

[raisinbear]

Thanks a lot, the duplicate key errors are what I was looking for. I've seen this, too, however, I thought that was due to me experimenting with the dev environment 😅. But there seems to be more going on. It looks like some database transactions were interrupted by a timeout or the microservices container crashing and restarting. Do you have machine learning enabled? It might be worth disabling it in the env file setting MACHINE_LEARNING_URL=false. I see your hardware is an Intel compute stick. If it's a 2 GB variant or there are other resource hungry processes running, your machine may run out of resources during upload processing, especially since the concurrency of ML jobs is 3 if I'm not mistaken. But even if it helped during new uploads, this wouldn't do anything for the now missing images. If this is like the issue I experienced (as mentioned probably my fault back then), there are invalid entries in your database preventing the missing images from being uploaded because there is already a record for them in the database. I mainly see three options:

1. Most sensible: See if one of the devs here has a better idea. Also, my conclusion may be completely wrong.
2. Reset the immich database and upload folder, losing all data in the instance and start over, hoping that disabling ML will keep new database errors from occuring. You would probably still need to regenerate the thumbnails afterwards until this other issue has been fixed.
3. Most discouraged: Manually delete the invalid entries from the database. I have been using a script to accomplish that, but I'm extremely hesitant to post it, since I don't know what other errors might come up for you in the future that can't easily be traced back to this manipulation.

[Isti6github]

Thanks a lot, the duplicate key errors are what I was looking for. I've seen this, too, however, I thought that was due to me experimenting with the dev environment 😅.

But there seems to be more going on. It looks like some database transactions were interrupted by a timeout or the microservices container crashing and restarting. Do you have machine learning enabled? It might be worth disabling it in the env file setting MACHINE_LEARNING_URL=false. I see your hardware is an Intel compute stick. If it's a 2 GB variant or there are other resource hungry processes running, your machine may run out of resources during upload processing, especially since the concurrency of ML jobs is 3 if I'm not mistaken.

But even if it helped during new uploads, this wouldn't do anything for the now missing images. If this is like the issue I experienced (as mentioned probably my fault back then), there are invalid entries in your database preventing the missing images from being uploaded because there is already a record for them in the database. I mainly see three options:

1. Most sensible: See if one of the devs here has a better idea. Also, my conclusion may be completely wrong.

2. Reset the immich database and upload folder, losing all data in the instance and start over, hoping that disabling ML will keep new database errors from occuring. You would probably still need to regenerate the thumbnails afterwards until this other issue has been fixed.

3. Most discouraged: Manually delete the invalid entries from the database. I have been using a script to accomplish that, but I'm extremely hesitant to post it, since I don't know what other errors might come up for you in the future that can't easily be traced back to this manipulation.

Thanks the ideas, I wait for some update, maybe fix it in the future. I use a 4gb compute stick with 4gb swap, same time just deluge and jellyfin run, but when I made the backup I not use jellyfin. What will happen if I disable ML? What the bennefit to use it?

[raisinbear]

It’s for object recognition in your images. But it’s a different container and you seem to have sufficient memory, so chances are rather slim this is the issue. If you can source the logs for the other containers in .txt format, I’d upload them just in case. There might be a hint what’s going on.

[Isti6github]

It’s for object recognition in your images. But it’s a different container and you seem to have sufficient memory, so chances are rather slim this is the issue. If you can source the logs for the other containers in .txt format, I’d upload them just in case. There might be a hint what’s going on.

I attached all, but I cant check the typsense and proxy (got error) _immich_postgres_logs.txt _immich_machine_learning_logs.txt _immich_web_logs.txt _immich_redis_logs.txt _immich_server_logs.txt _immich_microservices_logs.txt

[paveldudka]

I think Im facing the same issue. See my post in Discord: https://discord.com/channels/979116623879368755/1096683207539314899/1096683207539314899

In my case what I noticed is that number of photos that Immich is trying to back up is actually smaller than originally requested. In other words, sometimes "broken cloud" icon may indeed appear due to failed thumbnail creation job. In this case re-running a job manually helps. However, in my case actual image was probably not even included in a backup to begin with, so I don't think there is even a record in a DB for it. But that needs checking - I need to figure out a good way to cross-reference missing images and DB records

[alextran1502]

Hello is this issue still relevance? if not, can you help me close it?

[joss94]

Hi ! I still see this issue on my install (v1.91.4).

I observed that some of my recent photos were not updating because I had forgotten to set the "client_max_body_size" parameter on my nginx server. I fixed that and they got online fine.

Could it be that some older photos were not uploaded because of that, and are now not considered as candidates for backup any more?

[kerhbal]

hi I can confirm this still happens when I installed 1.99.0. my case:

• all photos are backed up to iCloud, and I have the originals on iPhone.
• "ignore photos backed up to iCloud" is turned off in immich
• among 15k photos, some have the icon of not uploading to cloud, despite the backup page shows 0 remaining
• and if I try to manually upload the photos, they can still be uploaded to server, and icon will turn to uploaded. I guess this means it's not an icon logic issue, there were not uploaded at first place

btw I created a discussion for a feature request on filtering by sync status on mobile https://github.com/immich-app/immich/discussions/8363

---

edit: the photos have problem are from a shared album, which means I guess the original photo is not on my phone

[alextran1502]

@kerhbal, can you try double pull to refresh or log out and log back in? I am pretty sure it it backup on the server already. The sync process might have bugs in it

[kerhbal]

@alextran1502 sorry I forgot that I already found the issue. those photos are from a shared album, probably that is the reason for the behaviour.

[alextran1502]

@kerhbal I believe so as we are not handling assets from Shared Album very well yet