immich icon indicating copy to clipboard operation
immich copied to clipboard

Published 20 hours ago • verified publisher icon immich-app

Files that are offline (deleted or missing) do not show in the trash folder.

Open kidroca opened this issue 1 year ago • 4 comments

The bug

As per release notes for v1.116 for external libraries:

Files that are offline (deleted or missing) now get hidden from the timeline and will show in the trash folder.

Immich identified 6000+ files as duplicates, I downloaded the list via an API call and used it to automatically move away most of the duplicates automatically

After that I've triggered a new scan for the external library, but non of the files appear in the trash folder - it's empty. Reviewing my duplicates and opening the full image I see an "Asset Offline" warning

This external library asset is no longer found on disk and has been moved to trash. If the file was moved within the library, check your timeline for the new corresponding asset. To restore this asset, please ensure that the file path below can be accessed by Immich and scan the library.

  1. Removed assets from external library are not appearing in trash
  2. Assets are still considered for duplicate review, because there's no way to remove them from Immich

I expected to see them in trash and empty the trash to remove all

A potential workaround might be to find all the asset IDs for the stuff I moved away and use the API to delete assets by ID

The OS that Immich Server is running on

ghcr.io/immich-app/immich-server:release

Version of Immich Server

v1.116.2

Version of Immich Mobile App

v1.0.0 (unused)

Platform with the issue

  • [X] Server
  • [X] Web
  • [ ] Mobile

Your docker-compose.yml content

#
# WARNING: Make sure to use the docker-compose.yml of the current release:
#
# https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml
#
# The compose file on main may not be compatible with the latest release.
#

name: immich

services:
  immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    extends:
      file: hwaccel.transcoding.yml
      service: cpu # set to one of [cpu, nvenc, quicksync, rkmpp, vaapi, vaapi-wsl] for accelerated transcoding
    volumes:
      # Do not edit the next line. If you want to change the media storage location on your system, edit the value of UPLOAD_LOCATION in the .env file
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /etc/localtime:/etc/localtime:ro
      - "/mnt/fastShare/kidroca/Google_Photos_Takeout:/mnt/family/kidroca/Google_Photos_Takeout"
      - "/mnt/fastShare/kidroca/PhotoSync:/mnt/family/kidroca/PhotoSync"
      - "/mnt/d/Nadya_Google_Photos_Takeout:/mnt/family/nadeto/Google_Photos_Takeout"
      - "/mnt/fastShare/nadeto/PhotoSync:/mnt/family/nadeto/PhotoSync"
    env_file:
      - .env
    ports:
      - 2283:3001
    depends_on:
      - redis
      - database
    restart: always
    healthcheck:
      disable: false

  immich-machine-learning:
    container_name: immich_machine_learning
    # For hardware acceleration, add one of -[armnn, cuda, openvino] to the image tag.
    # Example tag: ${IMMICH_VERSION:-release}-cuda
    image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}-cuda
    extends: # uncomment this section for hardware acceleration - see https://immich.app/docs/features/ml-hardware-acceleration
      file: hwaccel.ml.yml
      service: cuda # set to one of [cpu, armnn, cuda, openvino, openvino-wsl] for accelerated inference - use the `-wsl` version for WSL2 where applicable
    volumes:
      - model-cache:/cache
    env_file:
      - .env
    restart: always
    healthcheck:
      disable: false

  redis:
    container_name: immich_redis
    image: docker.io/redis:6.2-alpine@sha256:2d1463258f2764328496376f5d965f20c6a67f66ea2b06dc42af351f75248792
    healthcheck:
      test: redis-cli ping || exit 1
    restart: always

  database:
    container_name: immich_postgres
    image: docker.io/tensorchord/pgvecto-rs:pg14-v0.2.0@sha256:90724186f0a3517cf6914295b5ab410db9ce23190a2d9d0b9dd6463e3fa298f0
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_DB: ${DB_DATABASE_NAME}
      POSTGRES_INITDB_ARGS: '--data-checksums'
    volumes:
      # Do not edit the next line. If you want to change the database storage location on your system, edit the value of DB_DATA_LOCATION in the .env file
      - ${DB_DATA_LOCATION}:/var/lib/postgresql/data
    healthcheck:
      test: pg_isready --dbname='${DB_DATABASE_NAME}' --username='${DB_USERNAME}' || exit 1; Chksum="$$(psql --dbname='${DB_DATABASE_NAME}' --username='${DB_USERNAME}' --tuples-only --no-align --command='SELECT COALESCE(SUM(checksum_failures), 0) FROM pg_stat_database')"; echo "checksum failure count is $$Chksum"; [ "$$Chksum" = '0' ] || exit 1
      interval: 5m
      start_interval: 30s
      start_period: 5m
    command: ["postgres", "-c", "shared_preload_libraries=vectors.so", "-c", 'search_path="$$user", public, vectors', "-c", "logging_collector=on", "-c", "max_wal_size=2GB", "-c", "shared_buffers=512MB", "-c", "wal_compression=on"]
    restart: always

volumes:
  model-cache:

Your .env content

# You can find documentation for all the supported env variables at https://immich.app/docs/install/environment-variables

# The location where your uploaded files are stored
UPLOAD_LOCATION="/mnt/immich"
# The location where your database files are stored
DB_DATA_LOCATION=./postgres

# To set a timezone, uncomment the next line and change Etc/UTC to a TZ identifier from this list: https://en.wikipedia.org/wiki/List_of_tz_database_time_zones#List
# TZ=Etc/UTC

# The Immich version to use. You can pin this to a specific version like "v1.71.0"
IMMICH_VERSION=release

TZ=Europe/Sofia

# Connection secret for postgres. You should change it to a random password
# Please use only the characters `A-Za-z0-9`, without special characters or spaces
DB_PASSWORD=****

# The values below this line do not need to be changed
###################################################################################
DB_USERNAME=postgres
DB_DATABASE_NAME=immich

Reproduction steps

  1. Add external library
  2. Scan external library
  3. Remove files from external library
  4. Scan external library
  5. Go to trash

Observe: offline files (indexed files now missing in ext library) are not listed in trash

Relevant log output

[Nest] 7  - 09/30/2024, 12:00:00 AM     LOG [Microservices:LibraryService] Refreshing library 072fe9b3-c7eb-4551-b6ee-6d71898d2f37 for new assets
[Nest] 7  - 09/30/2024, 12:00:00 AM     LOG [Microservices:LibraryService] Refreshing library d0bdd3ea-0b53-4093-bfca-d105c646f28c for new assets
[Nest] 7  - 09/30/2024, 12:00:00 AM     LOG [Microservices:LibraryService] Scanning library d0bdd3ea-0b53-4093-bfca-d105c646f28c for removed assets
[Nest] 7  - 09/30/2024, 12:00:00 AM     LOG [Microservices:LibraryService] Scanning library 072fe9b3-c7eb-4551-b6ee-6d71898d2f37 for removed assets
[Nest] 7  - 09/30/2024, 12:00:00 AM     LOG [Microservices:LibraryService] Refreshing library d0bdd3ea-0b53-4093-bfca-d105c646f28c for new assets
[Nest] 7  - 09/30/2024, 12:00:00 AM     LOG [Microservices:LibraryService] Scanning library d0bdd3ea-0b53-4093-bfca-d105c646f28c for removed assets
[Nest] 7  - 09/30/2024, 12:00:00 AM     LOG [Microservices:LibraryService] Refreshing library 072fe9b3-c7eb-4551-b6ee-6d71898d2f37 for new assets
[Nest] 7  - 09/30/2024, 12:00:00 AM     LOG [Microservices:LibraryService] Scanning library 072fe9b3-c7eb-4551-b6ee-6d71898d2f37 for removed assets
[Nest] 7  - 09/30/2024, 12:00:24 AM     LOG [Microservices:LibraryService] Finished queueing check of 41157 assets for library 072fe9b3-c7eb-4551-b6ee-6d71898d2f37
[Nest] 7  - 09/30/2024, 12:00:25 AM     LOG [Microservices:LibraryService] Finished queueing check of 41157 assets for library 072fe9b3-c7eb-4551-b6ee-6d71898d2f37
[Nest] 7  - 09/30/2024, 12:00:29 AM     LOG [Microservices:LibraryService] Finished queueing check of 79691 assets for library d0bdd3ea-0b53-4093-bfca-d105c646f28c
[Nest] 7  - 09/30/2024, 12:00:30 AM     LOG [Microservices:LibraryService] Finished queueing check of 79691 assets for library d0bdd3ea-0b53-4093-bfca-d105c646f28c
[Nest] 7  - 09/30/2024, 2:00:00 AM     LOG [Microservices:LibraryService] Refreshing library d0bdd3ea-0b53-4093-bfca-d105c646f28c for new assets
[Nest] 7  - 09/30/2024, 2:00:00 AM     LOG [Microservices:LibraryService] Refreshing library 072fe9b3-c7eb-4551-b6ee-6d71898d2f37 for new assets
[Nest] 7  - 09/30/2024, 2:00:00 AM     LOG [Microservices:LibraryService] Refreshing library d0bdd3ea-0b53-4093-bfca-d105c646f28c for new assets
[Nest] 7  - 09/30/2024, 2:00:00 AM     LOG [Microservices:LibraryService] Refreshing library 072fe9b3-c7eb-4551-b6ee-6d71898d2f37 for new assets
[Nest] 7  - 09/30/2024, 2:00:00 AM     LOG [Microservices:LibraryService] Scanning library d0bdd3ea-0b53-4093-bfca-d105c646f28c for removed assets
[Nest] 7  - 09/30/2024, 2:00:05 AM     LOG [Microservices:LibraryService] Scanning library 072fe9b3-c7eb-4551-b6ee-6d71898d2f37 for removed assets
[Nest] 7  - 09/30/2024, 2:00:06 AM     LOG [Microservices:LibraryService] Scanning library d0bdd3ea-0b53-4093-bfca-d105c646f28c for removed assets
[Nest] 7  - 09/30/2024, 2:00:06 AM     LOG [Microservices:LibraryService] Scanning library 072fe9b3-c7eb-4551-b6ee-6d71898d2f37 for removed assets
[Nest] 7  - 09/30/2024, 2:00:25 AM     LOG [Microservices:LibraryService] Finished queueing check of 41157 assets for library 072fe9b3-c7eb-4551-b6ee-6d71898d2f37
[Nest] 7  - 09/30/2024, 2:00:25 AM     LOG [Microservices:LibraryService] Finished queueing check of 41157 assets for library 072fe9b3-c7eb-4551-b6ee-6d71898d2f37
[Nest] 7  - 09/30/2024, 2:00:30 AM     LOG [Microservices:LibraryService] Finished queueing check of 79691 assets for library d0bdd3ea-0b53-4093-bfca-d105c646f28c
[Nest] 7  - 09/30/2024, 2:00:30 AM     LOG [Microservices:LibraryService] Finished queueing check of 79691 assets for library d0bdd3ea-0b53-4093-bfca-d105c646f28c
[Nest] 7  - 09/30/2024, 4:00:00 AM     LOG [Microservices:LibraryService] Refreshing library d0bdd3ea-0b53-4093-bfca-d105c646f28c for new assets
[Nest] 7  - 09/30/2024, 4:00:00 AM     LOG [Microservices:LibraryService] Refreshing library 072fe9b3-c7eb-4551-b6ee-6d71898d2f37 for new assets
[Nest] 7  - 09/30/2024, 4:00:00 AM     LOG [Microservices:LibraryService] Refreshing library d0bdd3ea-0b53-4093-bfca-d105c646f28c for new assets
[Nest] 7  - 09/30/2024, 4:00:00 AM     LOG [Microservices:LibraryService] Refreshing library 072fe9b3-c7eb-4551-b6ee-6d71898d2f37 for new assets
[Nest] 7  - 09/30/2024, 4:00:00 AM     LOG [Microservices:LibraryService] Scanning library d0bdd3ea-0b53-4093-bfca-d105c646f28c for removed assets
[Nest] 7  - 09/30/2024, 4:00:05 AM     LOG [Microservices:LibraryService] Scanning library d0bdd3ea-0b53-4093-bfca-d105c646f28c for removed assets
[Nest] 7  - 09/30/2024, 4:00:05 AM     LOG [Microservices:LibraryService] Scanning library 072fe9b3-c7eb-4551-b6ee-6d71898d2f37 for removed assets
[Nest] 7  - 09/30/2024, 4:00:06 AM     LOG [Microservices:LibraryService] Scanning library 072fe9b3-c7eb-4551-b6ee-6d71898d2f37 for removed assets
[Nest] 7  - 09/30/2024, 4:00:24 AM     LOG [Microservices:LibraryService] Finished queueing check of 41157 assets for library 072fe9b3-c7eb-4551-b6ee-6d71898d2f37
[Nest] 7  - 09/30/2024, 4:00:25 AM     LOG [Microservices:LibraryService] Finished queueing check of 41157 assets for library 072fe9b3-c7eb-4551-b6ee-6d71898d2f37
[Nest] 7  - 09/30/2024, 4:00:29 AM     LOG [Microservices:LibraryService] Finished queueing check of 79691 assets for library d0bdd3ea-0b53-4093-bfca-d105c646f28c
[Nest] 7  - 09/30/2024, 4:00:30 AM     LOG [Microservices:LibraryService] Finished queueing check of 79691 assets for library d0bdd3ea-0b53-4093-bfca-d105c646f28c
[Nest] 7  - 09/30/2024, 4:02:37 AM     LOG [Microservices:LibraryService] Importing new library asset: /mnt/family/nadeto/PhotoSync/2024/09/IMG_9007.PNG
[Nest] 7  - 09/30/2024, 4:02:37 AM    WARN [Microservices:MetadataService] Asset 220e045c-4051-4043-b346-b3a3a8c9ef78 has no time zone information
[Nest] 7  - 09/30/2024, 4:02:38 AM     LOG [Microservices:MediaService] Successfully generated WEBP image preview for asset /mnt/family/nadeto/PhotoSync/2024/09/IMG_9007.PNG
[Nest] 7  - 09/30/2024, 4:02:38 AM     LOG [Microservices:MediaService] Successfully generated WEBP image thumbnail for asset /mnt/family/nadeto/PhotoSync/2024/09/IMG_9007.PNG
[Nest] 7  - 09/30/2024, 6:00:00 AM     LOG [Microservices:LibraryService] Refreshing library d0bdd3ea-0b53-4093-bfca-d105c646f28c for new assets
[Nest] 7  - 09/30/2024, 6:00:00 AM     LOG [Microservices:LibraryService] Refreshing library d0bdd3ea-0b53-4093-bfca-d105c646f28c for new assets
[Nest] 7  - 09/30/2024, 6:00:00 AM     LOG [Microservices:LibraryService] Refreshing library 072fe9b3-c7eb-4551-b6ee-6d71898d2f37 for new assets
[Nest] 7  - 09/30/2024, 6:00:00 AM     LOG [Microservices:LibraryService] Refreshing library 072fe9b3-c7eb-4551-b6ee-6d71898d2f37 for new assets
[Nest] 7  - 09/30/2024, 6:00:00 AM     LOG [Microservices:LibraryService] Scanning library d0bdd3ea-0b53-4093-bfca-d105c646f28c for removed assets
[Nest] 7  - 09/30/2024, 6:00:00 AM     LOG [Microservices:LibraryService] Scanning library d0bdd3ea-0b53-4093-bfca-d105c646f28c for removed assets
[Nest] 7  - 09/30/2024, 6:00:00 AM     LOG [Microservices:LibraryService] Scanning library 072fe9b3-c7eb-4551-b6ee-6d71898d2f37 for removed assets
[Nest] 7  - 09/30/2024, 6:00:00 AM     LOG [Microservices:LibraryService] Scanning library 072fe9b3-c7eb-4551-b6ee-6d71898d2f37 for removed assets
[Nest] 7  - 09/30/2024, 6:00:24 AM     LOG [Microservices:LibraryService] Finished queueing check of 41157 assets for library 072fe9b3-c7eb-4551-b6ee-6d71898d2f37
[Nest] 7  - 09/30/2024, 6:00:25 AM     LOG [Microservices:LibraryService] Finished queueing check of 41157 assets for library 072fe9b3-c7eb-4551-b6ee-6d71898d2f37
[Nest] 7  - 09/30/2024, 6:00:30 AM     LOG [Microservices:LibraryService] Finished queueing check of 79692 assets for library d0bdd3ea-0b53-4093-bfca-d105c646f28c
[Nest] 7  - 09/30/2024, 6:00:31 AM     LOG [Microservices:LibraryService] Finished queueing check of 79692 assets for library d0bdd3ea-0b53-4093-bfca-d105c646f28c
[Nest] 7  - 09/30/2024, 8:00:00 AM     LOG [Microservices:LibraryService] Refreshing library d0bdd3ea-0b53-4093-bfca-d105c646f28c for new assets
[Nest] 7  - 09/30/2024, 8:00:00 AM     LOG [Microservices:LibraryService] Refreshing library d0bdd3ea-0b53-4093-bfca-d105c646f28c for new assets
[Nest] 7  - 09/30/2024, 8:00:00 AM     LOG [Microservices:LibraryService] Refreshing library 072fe9b3-c7eb-4551-b6ee-6d71898d2f37 for new assets
[Nest] 7  - 09/30/2024, 8:00:00 AM     LOG [Microservices:LibraryService] Scanning library d0bdd3ea-0b53-4093-bfca-d105c646f28c for removed assets
[Nest] 7  - 09/30/2024, 8:00:00 AM     LOG [Microservices:LibraryService] Refreshing library 072fe9b3-c7eb-4551-b6ee-6d71898d2f37 for new assets
[Nest] 7  - 09/30/2024, 8:00:00 AM     LOG [Microservices:LibraryService] Scanning library 072fe9b3-c7eb-4551-b6ee-6d71898d2f37 for removed assets
[Nest] 7  - 09/30/2024, 8:00:05 AM     LOG [Microservices:LibraryService] Scanning library 072fe9b3-c7eb-4551-b6ee-6d71898d2f37 for removed assets
[Nest] 7  - 09/30/2024, 8:00:05 AM     LOG [Microservices:LibraryService] Scanning library d0bdd3ea-0b53-4093-bfca-d105c646f28c for removed assets
[Nest] 7  - 09/30/2024, 8:00:24 AM     LOG [Microservices:LibraryService] Finished queueing check of 41157 assets for library 072fe9b3-c7eb-4551-b6ee-6d71898d2f37
[Nest] 7  - 09/30/2024, 8:00:25 AM     LOG [Microservices:LibraryService] Finished queueing check of 41157 assets for library 072fe9b3-c7eb-4551-b6ee-6d71898d2f37
[Nest] 7  - 09/30/2024, 8:00:30 AM     LOG [Microservices:LibraryService] Finished queueing check of 79692 assets for library d0bdd3ea-0b53-4093-bfca-d105c646f28c
[Nest] 7  - 09/30/2024, 8:00:30 AM     LOG [Microservices:LibraryService] Finished queueing check of 79692 assets for library d0bdd3ea-0b53-4093-bfca-d105c646f28c
[Nest] 7  - 09/30/2024, 10:00:00 AM     LOG [Microservices:LibraryService] Refreshing library d0bdd3ea-0b53-4093-bfca-d105c646f28c for new assets
[Nest] 7  - 09/30/2024, 10:00:00 AM     LOG [Microservices:LibraryService] Refreshing library 072fe9b3-c7eb-4551-b6ee-6d71898d2f37 for new assets
[Nest] 7  - 09/30/2024, 10:00:00 AM     LOG [Microservices:LibraryService] Refreshing library d0bdd3ea-0b53-4093-bfca-d105c646f28c for new assets
[Nest] 7  - 09/30/2024, 10:00:00 AM     LOG [Microservices:LibraryService] Refreshing library 072fe9b3-c7eb-4551-b6ee-6d71898d2f37 for new assets
[Nest] 7  - 09/30/2024, 10:00:00 AM     LOG [Microservices:LibraryService] Scanning library d0bdd3ea-0b53-4093-bfca-d105c646f28c for removed assets
[Nest] 7  - 09/30/2024, 10:00:00 AM     LOG [Microservices:LibraryService] Scanning library 072fe9b3-c7eb-4551-b6ee-6d71898d2f37 for removed assets
[Nest] 7  - 09/30/2024, 10:00:03 AM     LOG [Microservices:LibraryService] Scanning library d0bdd3ea-0b53-4093-bfca-d105c646f28c for removed assets
[Nest] 7  - 09/30/2024, 10:00:05 AM     LOG [Microservices:LibraryService] Scanning library 072fe9b3-c7eb-4551-b6ee-6d71898d2f37 for removed assets
[Nest] 7  - 09/30/2024, 10:00:26 AM     LOG [Microservices:LibraryService] Finished queueing check of 41157 assets for library 072fe9b3-c7eb-4551-b6ee-6d71898d2f37
[Nest] 7  - 09/30/2024, 10:00:26 AM     LOG [Microservices:LibraryService] Finished queueing check of 41157 assets for library 072fe9b3-c7eb-4551-b6ee-6d71898d2f37
[Nest] 7  - 09/30/2024, 10:00:32 AM     LOG [Microservices:LibraryService] Finished queueing check of 79692 assets for library d0bdd3ea-0b53-4093-bfca-d105c646f28c
[Nest] 7  - 09/30/2024, 10:00:32 AM     LOG [Microservices:LibraryService] Finished queueing check of 79692 assets for library d0bdd3ea-0b53-4093-bfca-d105c646f28c

Additional information

No response

kidroca avatar Sep 30 '24 09:09 kidroca

@kidroca Can this be reproducible with a single external library?

@zackpollard, Can you help take a look at this issue? Thank you

alextran1502 avatar Sep 30 '24 10:09 alextran1502

I tried moving files around and renaming them (to create a new file and offline the old path) and they always end up in trash for me

zackpollard avatar Sep 30 '24 11:09 zackpollard

@alextran1502

@kidroca Can this be reproducible with a single external library?

I've added all paths under the same external library, so any duplicates can be detected:

      - "/mnt/fastShare/kidroca/Google_Photos_Takeout:/mnt/family/kidroca/Google_Photos_Takeout"
      - "/mnt/fastShare/kidroca/PhotoSync:/mnt/family/kidroca/PhotoSync"
      - "/mnt/d/Nadya_Google_Photos_Takeout:/mnt/family/nadeto/Google_Photos_Takeout"
      - "/mnt/fastShare/nadeto/PhotoSync:/mnt/family/nadeto/PhotoSync"

But originally, the paths below were added to a separate external library

      - "/mnt/d/Nadya_Google_Photos_Takeout:/mnt/family/nadeto/Google_Photos_Takeout"
      - "/mnt/fastShare/nadeto/PhotoSync:/mnt/family/nadeto/PhotoSync"

Initially I created 2 external libraries, but later realized that this won't detect duplicates across different external libraries

I didn't want to lose all indexing / thumbnails / face recognition of the 2nd external library, so I didn't remove it.

Now I have one external library that contains everything and a 2nd external library, that contains a subset.

I guess I should have started with a single library and split into 2 only after sorting out the duplicates

In any case I've removed about 6000 files flagged as duplicates by Immich. Reviewing individual files via the duplicates utility confirms they are offline, but the /trash view is empty

kidroca avatar Sep 30 '24 11:09 kidroca

I've just removed a bunch of videos outside of Immich and rescanned libraries and the removed videos do appear in the trash

I don't think these videos were flagged as duplicates by Immich, could this be the problem?

The assets flagged as duplicates don't appear in trash, because they are flagged as duplicates and somehow this takes priority over listing them in trash?

kidroca avatar Sep 30 '24 11:09 kidroca

I think I've run into the same/similar problem on v1.121.0.

For my immich_server container, the volume mapping for the external library is as follows:

/ExtHDD/External-Libraries:/usr/src/app/external:ro

I had a bunch of assets in /ExtHDD/External-Libraries/Old-Folder that were scanned into Immich.

I then renamed it to /ExtHDD/External-Libraries/New-Folder and all those assets were scanned into Immich. I also ran the Library job.

However, none of the Old-Folder assets have been moved to the trash or marked as offline. I know they are not offline because I tried to use Immich Folder Album Creator to delete offline assets, and it said it didn't find any. Unless that tool is having issues finding offline assets.

I see assets in Immich (web UI) that have the Old-Folder path and duplicate assets with the New-Folder path.

Why aren't the Old-Folder assets set to offline and in the trash?

nothing2obvi avatar Dec 02 '24 18:12 nothing2obvi

I think I've run into the same/similar problem on v1.121.0.

For my immich_server container, the volume mapping for the external library is as follows:

/ExtHDD/External-Libraries:/usr/src/app/external:ro

I had a bunch of assets in /ExtHDD/External-Libraries/Old-Folder that were scanned into Immich.

I then renamed it to /ExtHDD/External-Libraries/New-Folder and all those assets were scanned into Immich. I also ran the Library job.

However, none of the Old-Folder assets have been moved to the trash or marked as offline. I know they are not offline because I tried to use Immich Folder Album Creator to delete offline assets, and it said it didn't find any. Unless that tool is having issues finding offline assets.

I see assets in Immich (web UI) that have the Old-Folder path and duplicate assets with the New-Folder path.

Why aren't the Old-Folder assets set to offline and in the trash?

If you check the immich trash folder, are the assets there? If not, can you see the assets in the timeline?

You can also directly check if an asset is offline by inspecting its json.

Check the URL of the page and note the UUID

image

In Chrome F12, go to the Network tab and click F5.

Select the request corresponding to the asset

image

In the Response tab, check the asset info json:

image

etnoy avatar Dec 18 '24 21:12 etnoy

Thanks for your reply. I will find some time to test over the next week.

nothing2obvi avatar Dec 21 '24 23:12 nothing2obvi

Ver : V1.124.0

External Library (read only)

I have removed the file in NAS. And immich can detected it and show in Trash View. But when I try to click "Empty trash". It not work and the image still show in Trash View.

ffchung avatar Jan 08 '25 06:01 ffchung

We currently have it so that emptying trash does not remove offline files. That's a design decision as this point though it might change in the future. Offline files will be removed after 30 days. I know the author of immich-power-tools has a feature to delete offline files at this point if you really want to

etnoy avatar Jan 08 '25 07:01 etnoy

@etnoy Thanks for suggestion. But you just give me a idea. If I change the Trash Setting to 7 days, will it remove offline files early?

ffchung avatar Jan 08 '25 08:01 ffchung

@etnoy Thanks for suggestion. But you just give me a idea. If I change the Trash Setting to 7 days, will it remove offline files early?

That is correct, it will then remove them after 7 days

etnoy avatar Jan 08 '25 21:01 etnoy

Closing this issue for now, if it's still an issue in the latest version, please reopen and provide information that I requested above.

etnoy avatar Jan 08 '25 21:01 etnoy

I'm in the same boat. I have images that moved locations within the same external library and are picked up as duplicates. I can see them in the timeline, as well as in the duplicate utility. Their JSON is showing 'isOffline: true'.

The offline assets do not show up in the trash, and emptying the trash after the grace period (30 days in my case) does not remove anything.

I've used various tools (like Thoroslives' immich_remove_offline_files script), but none actually find any offline files to remove.

Note that I've had this issue for over 2 months, across many versions of Immich. Currently running v1.124.2.

jpiscaer avatar Jan 09 '25 16:01 jpiscaer

I'm in the same boat. I have images that moved locations within the same external library and are picked up as duplicates. I can see them in the timeline, as well as in the duplicate utility. Their JSON is showing 'isOffline: true'.

The offline assets do not show up in the trash, and emptying the trash after the grace period (30 days in my case) does not remove anything.

I've used various tools (like Thoroslives' immich_remove_offline_files script), but none actually find any offline files to remove.

Note that I've had this issue for over 2 months, across many versions of Immich. Currently running v1.124.2.

Would you be willing to post the full json for the problematic assets? Redact any sensitive data

etnoy avatar Jan 09 '25 19:01 etnoy

Here's the full JSON for one of those files:

{
    "id": "7f0f5370-a132-4842-b947-932885e7fcc9",
    "deviceAssetId": "file130.jpg",
    "ownerId": "[redacted]",
    "owner": {
        "id": "[redacted]",
        "email": "[redacted]",
        "name": "[redacted]",
        "profileImagePath": "",
        "avatarColor": "red",
        "profileChangedAt": "2024-11-05T14:26:24.331Z"
    },
    "deviceId": "Library Import",
    "libraryId": "8469af06-5a1d-4d74-bd14-797e33a2ee76",
    "type": "IMAGE",
    "originalPath": "/mnt/zfs-pool-18tb/Pictures/oldpath/file130.jpg",
    "originalFileName": "file130.jpg",
    "originalMimeType": "image/jpeg",
    "thumbhash": "yScGDYRfZpZIpXh5lrd5VoaAcsC3",
    "fileCreatedAt": "2013-01-18T21:12:43.000Z",
    "fileModifiedAt": "2013-01-18T21:12:44.000Z",
    "localDateTime": "2013-01-18T22:12:43.000Z",
    "updatedAt": "2024-12-04T10:50:12.591Z",
    "isFavorite": false,
    "isArchived": false,
    "isTrashed": false,
    "duration": "0:00:00.00000",
    "exifInfo": {
        "make": "Canon",
        "model": "Canon EOS 20D",
        "exifImageWidth": 2628,
        "exifImageHeight": 1752,
        "fileSizeInByte": 416336,
        "orientation": "1",
        "dateTimeOriginal": "2013-01-18T21:12:43.000Z",
        "modifyDate": "2013-01-18T21:12:43.000Z",
        "timeZone": null,
        "lensModel": null,
        "fNumber": 4.5,
        "focalLength": 21,
        "iso": 100,
        "exposureTime": "1/10",
        "latitude": null,
        "longitude": null,
        "city": null,
        "state": null,
        "country": null,
        "description": "",
        "projectionType": null,
        "rating": null
    },
    "livePhotoVideoId": null,
    "tags": [],
    "people": [
        {
            "id": "b8ffdc38-338a-44fa-b852-588b19b5c05a",
            "name": "",
            "birthDate": null,
            "thumbnailPath": "",
            "isHidden": false,
            "updatedAt": "2024-11-09T23:04:16.854Z",
            "faces": [
                {
                    "id": "848ef03b-27a9-44e6-be0e-e3b9c05df1f4",
                    "imageHeight": 1440,
                    "imageWidth": 2160,
                    "boundingBoxX1": 1152,
                    "boundingBoxX2": 1257,
                    "boundingBoxY1": 252,
                    "boundingBoxY2": 393,
                    "sourceType": "machine-learning"
                }
            ]
        }
    ],
    "unassignedFaces": [],
    "checksum": "/aQJgQzodyzkoU4BuQPmYkmy+xY=",
    "stack": null,
    "isOffline": true,
    "hasMetadata": true,
    "duplicateId": "543910a4-5184-4d73-82db-44569c34f377",
    "resized": true
}

jpiscaer avatar Jan 10 '25 08:01 jpiscaer

What's most weird to me is that none of the 3rd party tools (like Thoroslives' immich_remove_offline_files script) pick up on the offline files:

Image

jpiscaer avatar Jan 17 '25 20:01 jpiscaer

It seems weird that is has isOffline: true but isTrashed: false. IIRC it's supposed to be either both or none.

Take a database backup, then connect to your database with psql or similar SQL tool. Then run an update on the assets column to set all assets to isOffline = true and report back. It's likely set in some limbo state so that neither immich nor other tools are understanding the issue

etnoy avatar Jan 18 '25 00:01 etnoy

Yeah, figured that Immich doesn't have a way in the UI to deal with files that are offline but not in trash.

I used the following code to remove files that were in the old folder:

$ sudo docker exec -it immich_postgres bash
# su postgres
$ psql

\c immich
SELECT * FROM assets WHERE "originalPath" LIKE '%/old folder/%';
DELETE FROM assets WHERE "originalPath" LIKE '%/old folder/%';

jpiscaer avatar Jan 18 '25 18:01 jpiscaer

What's most weird to me is that none of the 3rd party tools (like Thoroslives' immich_remove_offline_files script) pick up on the offline files:

These tools (mine as well) rely on the search returning offline images. For quite some time, the search no longer returns assets that are offline and/or trashed.

Salvoxia avatar Feb 13 '25 06:02 Salvoxia