[BUG] Slow preview and pictures/video load from device memory

[stefano99]

The bug

When loading from memory, the app starts to lag and the time required to load the previews goes to around 10s, 20s at times.

It happened as soon as I updated the app to the 1.51.0 version.

Resetting anad reinstalling does nothing.

That's how it looks like when I open the app, or when I scroll down the gallery:

Downgrading back to 1.50.0 brings back the usual speed. Also, opening the gallery from another device which doesn't sync any folder or from the web doesn't present this behavior.

That's the log file: Immich_log_2023-03-30T192314.732190.csv

The OS that Immich Server is running on

Ubuntu 20.04 HWE (Linux 5.15.0)

Version of Immich Server

1.52.0

Version of Immich Mobile App

1.52.0

Platform with the issue

• [ ] Server
• [ ] Web
• [X] Mobile

Your docker-compose.yml content

version: "3.7"

volumes:
  pgdata:
  model-cache:
  tsdata:

networks:
  t2_proxy:
    external: true
  immich:

services:
  immich-server:
    container_name: immich_server
    image: altran1502/immich-server:release
    entrypoint: [ "/bin/sh", "./start-server.sh" ]
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
    env_file:
      - .env
    environment:
      - NODE_ENV=production
    depends_on:
      - redis
      - database
      - typesense
    restart: always
    networks:
      - immich

  immich-microservices:
    container_name: immich_microservices
    image: altran1502/immich-server:release
    entrypoint: [ "/bin/sh", "./start-microservices.sh" ]
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
    env_file:
      - .env
    environment:
      - NODE_ENV=production
    depends_on:
      - redis
      - database
      - typesense
    restart: always
    networks:
      - immich

  immich-machine-learning:
    container_name: immich_machine_learning
    image: altran1502/immich-machine-learning:release
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - model-cache:/cache
    env_file:
      - .env
    environment:
      - NODE_ENV=production
    restart: always
    networks:
      - immich

  immich-web:
    container_name: immich_web
    image: altran1502/immich-web:release
    entrypoint: [ "/bin/sh", "./entrypoint.sh" ]
    env_file:
      - .env
    restart: always
    networks:
      - immich
  
  typesense:
    container_name: immich_typesense
    image: typesense/typesense:0.24.0
    environment:
      - TYPESENSE_API_KEY=${TYPESENSE_API_KEY}
      - TYPESENSE_DATA_DIR=/data
    logging:
      driver: none
    volumes:
      - tsdata:/data
    restart: always
    networks:
      - immich

  redis:
    container_name: immich_redis
    image: redis:6.2
    restart: always
    networks:
      - immich

  database:
    container_name: immich_postgres
    image: postgres:14
    env_file:
      - .env
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_DB: ${DB_DATABASE_NAME}
      PG_DATA: /var/lib/postgresql/data
    volumes:
      # - pgdata:/var/lib/postgresql/data
      - $DOCKEREXT/immich_docker/postgresql/data:/var/lib/postgresql/data
    restart: always
    networks:
      - immich

  immich-proxy:
    container_name: immich_proxy
    image: altran1502/immich-proxy:release
    environment:
      # Make sure these values get passed through from the env file
      - IMMICH_SERVER_URL
      - IMMICH_WEB_URL
    ports:
      - 2283:8080
    logging:
      driver: none
    depends_on:
      - immich-server
    restart: always
    networks:
      - t2_proxy
      - immich
    labels:
      - "traefik.enable=true"
      - "traefik.docker.network=t2_proxy"
      - "traefik.http.routers.immich-rtr.entrypoints=https"
      - "traefik.http.routers.immich.rule=Host(`photos.$DOMAINNAME`)"
      - "traefik.http.routers.immich.tls=true"
      - "traefik.http.routers.immich-rtr.service=immich-svc"
      - "traefik.http.services.immich-svc.loadbalancer.server.port=8080"
      - "traefik.http.routers.immich-rtr.middlewares=chain-no-auth@file"
    #   # - "traefik.http.routers.immich-rtr.middlewares=chain-nextcloud@file"
    # homepage labels
      - homepage.group=immich
      - homepage.name=immich
      # - homepage.icon=emby.png
      - homepage.href=https://photos.$DOMAINNAME
      - homepage.description="immich gallery/backup"

Your .env content

###################################################################################
# Database
###################################################################################

DB_HOSTNAME=immich_postgres
DB_USERNAME=...
DB_PASSWORD=...
DB_DATABASE_NAME=immich

# Optional Database settings:
# DB_PORT=5432

###################################################################################
# Redis
###################################################################################

REDIS_HOSTNAME=immich_redis

# Optional Redis settings:
# REDIS_PORT=6379
# REDIS_DBINDEX=0
# REDIS_PASSWORD=
# REDIS_SOCKET=

###################################################################################
# Upload File Config
###################################################################################

UPLOAD_LOCATION=/../..

###################################################################################
# Typesense
###################################################################################
TYPESENSE_API_KEY=
# TYPESENSE_ENABLED=false

###################################################################################
# Log message level - [simple|verbose]
###################################################################################

LOG_LEVEL=verbose

###################################################################################
# JWT SECRET
###################################################################################

# This JWT_SECRET is used to sign the authentication keys for user login
# You should set it to a long randomly generated value
# You can use this command to generate one: openssl rand -base64 128
JWT_SECRET=...

###################################################################################
# Reverse Geocoding
####################################################################################

# DISABLE_REVERSE_GEOCODING=false

# Reverse geocoding is done locally which has a small impact on memory usage
# This memory usage can be altered by changing the REVERSE_GEOCODING_PRECISION variable
# This ranges from 0-3 with 3 being the most precise
# 3 - Cities > 500 population: ~200MB RAM
# 2 - Cities > 1000 population: ~150MB RAM
# 1 - Cities > 5000 population: ~80MB RAM
# 0 - Cities > 15000 population: ~40MB RAM

# REVERSE_GEOCODING_PRECISION=3

####################################################################################
# WEB - Optional
####################################################################################

# Custom message on the login page, should be written in HTML form.
# For example PUBLIC_LOGIN_PAGE_MESSAGE="This is a demo instance of Immich.<br><br>Email: <i>[email protected]</i><br>Password: <i>demo</i>"

PUBLIC_LOGIN_PAGE_MESSAGE=...

# Alternative Service Addresses - Optional
#
# This is an advanced feature for users who may be running their immich services on different hosts.
# It will not change which address or port that services bind to within their containers, but it will change where other services look for their peers.
# Note: immich-microservices is bound to 3002, but no references are made
####################################################################################

# IMMICH_WEB_URL=http://immich-web:3000
# IMMICH_SERVER_URL=http://immich-server:3001
# IMMICH_MACHINE_LEARNING_URL=http://immich-machine-learning:3003

####################################################################################
# Alternative API's External Address - Optional
#
# This is an advanced feature used to control the public server endpoint returned to clients during Well-known discovery.
# You should only use this if you want mobile apps to access the immich API over a custom URL. Do not include trailing slash.
# NOTE: At this time, the web app will not be affected by this setting and will continue to use the relative path: /api
# Examples: http://localhost:3001, http://immich-api.example.com, etc
####################################################################################

#IMMICH_API_URL_EXTERNAL=http://localhost:3001

####################################################################################
# OAuth Setting - Optional
#
# These setting will enable OAuth login for your instance of Immich
# Folow the instructions in the page https://immich.app/docs/usage/oauth to set up your OAuth provider
####################################################################################

# OAUTH_ENABLED=false
# OAUTH_ISSUER_URL=
# OAUTH_CLIENT_ID=
# OAUTH_CLIENT_SECRET=
# OAUTH_BUTTON_TEXT=Login with OAuth
# OAUTH_AUTO_REGISTER=true
# OAUTH_SCOPE="openid profile email"

Reproduction steps

1.Reset the app
2.Delete the data manually
3.Log back in
4.Try to scroll through the timeline, jump around, etc
  This works fine.
5.From the settings, select the folders where there are the photos already synchronized.
6.Try to scroll through the timeline and jump around it.
  Now it shouldn't load the assets.

Additional information

No response

[alextran1502]

Can you help me confirm again which version of the mobile app you are running?

[stefano99]

The last one from the Play Store: 1.52.0 The slowness started with v1.51.0

[alextran1502]

Are you experiencing the same issue if you access the instance with local IP?

[stefano99]

Yes, it still happens Tried just now and that's the result:

I'm shooting in HEIF and HEVC formats, can that be the issue?

[alextran1502]

I'm shooting in HEIF and HEVC formats, can that be the issue?

It could be, since now it will try to read the local file to work with the offline mechanism. Before it loaded the data from the server

[stefano99]

Ok, I can try to shoot in jpeg for a couple of days and report back here if that improves the loading times.

[alextran1502]

On my iPhone, I am also shooting in HEIC, but I don't experience this issue. Which phone are you using?

[stefano99]

I'm using a Samsung Galaxy S21

Also, it heats up a lot while loading the pictures in the Immich app

[alextran1502]

Can you go to the upload screen, then go to the album selection screen and click on this icon?

Does it load instantly, or are you also facing the same issue you are seeing on the main timeline?

[stefano99]

Same thing: it takes a lot to load, and scrolling while loading makes it worse. I'm syncing also an album with only jpegs and it's the same.

That's how it looks

At the moment, don't know why, it's showing almost all the pictures as if they're only present on the server and not on the device. But looking at the backup information, it showsthe Total and Backup numbers as matching, and the same as before, like this:

If the pics are uploaded but not present locally, it shows the pictures instantly, and the same for the previews.

There's also this error popping out in the logs:

[alextran1502]

interesting log. We haven't seen that before. Can you export the log and attach it here?

[stefano99]

Sure! Here it is: Immich_log_2023-03-30T214349.208426.csv

Logging out and back in fixed it, btw

[stefano99]

Update after some time.

I moved the whole photo library away from the phone, and started shooting in jpg. It's roughly 50 pictures now, and it's (of course) faster. Still, it takes a bit to load the previews of the pictures from the local storage. Noticeably more than the ones stored only on the server, which load almost instantly.

[fyfrey]

can you check again with app version 1.55? we maybe did some improvements in the unreleased app version 1.54.1

[stefano99]

Until version 1.56 it was identical. Just updated to v1.57 and it looks like it's a bit faster! Reloading assets after I close a picture it's still the same, but it's slightly faster when launching the app.

[fyfrey]

thanks for checking again. at least the lazy loading seems to help a bit. are you still seeing loading times above 1 second?

and another test: can you create an album with only assets uploaded via web or CLI (not the app) and open that album in the app? does it load these assets fast?

[Obsidiannite21]

I'm also experience this issue. I reported it directly to @alextran1502 and @fyfrey via discord but wanted to put my info here to offer trouble shooting assistance if needed.

[stefano99]

@fyfrey sorry for the late reply, didn't see the notification!

Yes I'm still getting high loading times.

If the file isn't in the phone memory, it's almost instant in loading the previews, so everything that's uploaded through anything else, or is been deleted from the phone, is fast.

[fyfrey]

I think we can put in a workaround, to always load images from the server (configurable in the app settings). Unless we find the reason for this super slowness

edit: made this PR #3044 to provide the workaround as it seems there is no solution we can apply (since the issue is deep in one of Immich's dependencies)

[fyfrey]

I fear it's this bug: https://github.com/bumptech/glide/issues/4608

[Obsidiannite21]

Thank you for such a quick workaround!! @fyfrey

[stefano99]

@fyfrey I switched to jpeg after noticing this bug, and still getting the same behavior with the new images. I have to say, though, that with HEIC and HEVC it's even slower, so it's not strictly related to HEIF/HEIC format, but it does make it worse.

The workaround is great! Thank you!

[fyfrey]

Thanks for the info.

@stefano99 @Obsidiannite21 what resolution do the photos have? Is this like a 100 megapixel camera?

[stefano99]

@fyfrey 12MP for photos, 4k@30fps for videos (but it's the same even at 1080p)

The loading time increases as the number of elements on local storage increases, so if I remove photos and videos from the local storage, the first couple of pics I take load almost instantly. (The elements only on the server are not affected)

I'll keep an eye on that to see if there's a specific number of elements where it starts to lag.

[Obsidiannite21]

@fyfrey @stefano99 Same here - 12MP for photos. I also have a large library.

[stefano99]

I'm now at 56 between photos and videos and it's starting to lag a bit while loading or reloading the library after closing a pic/video or reopening the app.

[fyfrey]

With the newest release, you can enable the workaround in the advanced settings. Assets already uploaded should then load just fine

[stefano99]

@fyfrey thank you so much! It seems to be working flawlessly!

Thank you!

[Snyaify]

@fyfrey thank you so much for workaround. On my OnePlus 10 PRO with 12k photos in gallery it took eternity to load previews.

[fyfrey]

Could you all check whether another flutter app - Aves Gallery (https://github.com/deckerst/aves) has the same issues in loading thumbnails on your devices? If it works fine, we could probably look into how they do the image loading