immich
immich copied to clipboard
[BUG] 500 error when open "explore" on web and mobile things search.
The bug
Immich docker server log:
``` [31m[Nest] 1 - [39m04/25/2023, 9:21:52 AM [31m ERROR[39m [38;5;3m[ExceptionsHandler] [39m[31mRequest failed with HTTP code 404 | Server said: Could not find a field named `originalFileName` in the schema.[39m ObjectNotFound: Request failed with HTTP code 404 | Server said: Could not find a field named `originalFileName` in the schema. at ObjectNotFound.TypesenseError [as constructor] (/usr/src/app/node_modules/typesense/lib/Typesense/Errors/TypesenseError.js:23:28) at new ObjectNotFound (/usr/src/app/node_modules/typesense/lib/Typesense/Errors/ObjectNotFound.js:25:42) at ApiCall.customErrorForResponse (/usr/src/app/node_modules/typesense/lib/Typesense/ApiCall.js:338:21) at /usr/src/app/node_modules/typesense/lib/Typesense/ApiCall.js:199:98 at step (/usr/src/app/node_modules/typesense/lib/Typesense/ApiCall.js:33:23) at Object.next (/usr/src/app/node_modules/typesense/lib/Typesense/ApiCall.js:14:53) at step (/usr/src/app/node_modules/typesense/lib/Typesense/ApiCall.js:18:139) at Object.next (/usr/src/app/node_modules/typesense/lib/Typesense/ApiCall.js:14:53) at fulfilled (/usr/src/app/node_modules/typesense/lib/Typesense/ApiCall.js:5:58) at processTicksAndRejections (node:internal/process/task_queues:96:5) [31m[Nest] 1 - [39m04/25/2023, 9:22:00 AM [31m ERROR[39m [38;5;3m[ExceptionsHandler] [39m[31mRequest failed with HTTP code 404 | Server said: Could not find a field named `originalFileName` in the schema.[39m ObjectNotFound: Request failed with HTTP code 404 | Server said: Could not find a field named `originalFileName` in the schema. at ObjectNotFound.TypesenseError [as constructor] (/usr/src/app/node_modules/typesense/lib/Typesense/Errors/TypesenseError.js:23:28) at new ObjectNotFound (/usr/src/app/node_modules/typesense/lib/Typesense/Errors/ObjectNotFound.js:25:42) at ApiCall.customErrorForResponse (/usr/src/app/node_modules/typesense/lib/Typesense/ApiCall.js:338:21) at /usr/src/app/node_modules/typesense/lib/Typesense/ApiCall.js:199:98 at step (/usr/src/app/node_modules/typesense/lib/Typesense/ApiCall.js:33:23) at Object.next (/usr/src/app/node_modules/typesense/lib/Typesense/ApiCall.js:14:53) at step (/usr/src/app/node_modules/typesense/lib/Typesense/ApiCall.js:18:139) at Object.next (/usr/src/app/node_modules/typesense/lib/Typesense/ApiCall.js:14:53) at fulfilled (/usr/src/app/node_modules/typesense/lib/Typesense/ApiCall.js:5:58) at processTicksAndRejections (node:internal/process/task_queues:96:5) ```The OS that Immich Server is running on
Ubuntu 20.04.5 LTS x86_64(Py3.7.9)
Version of Immich Server
v1.54.1
Version of Immich Mobile App
v1.54.0 build.77
Platform with the issue
- [X] Server
- [X] Web
- [X] Mobile
Your docker-compose.yml content
docker-compose.yml content
version: "3.8"
services:
immich-server:
container_name: immich_server
image: ghcr.io/immich-app/immich-server:release
entrypoint: [ "/bin/sh", "./start-server.sh" ]
volumes:
- ${UPLOAD_LOCATION}:/usr/src/app/upload
env_file:
- .env
environment:
- NODE_ENV=production
depends_on:
- redis
- database
- typesense
restart: always
immich-microservices:
container_name: immich_microservices
image: ghcr.io/immich-app/immich-server:release
entrypoint: [ "/bin/sh", "./start-microservices.sh" ]
volumes:
- ${UPLOAD_LOCATION}:/usr/src/app/upload
env_file:
- .env
environment:
- NODE_ENV=production
depends_on:
- redis
- database
- typesense
restart: always
immich-machine-learning:
container_name: immich_machine_learning
image: ghcr.io/immich-app/immich-machine-learning:release
volumes:
- ${UPLOAD_LOCATION}:/usr/src/app/upload
- model-cache:/cache
env_file:
- .env
environment:
- NODE_ENV=production
restart: always
immich-web:
container_name: immich_web
image: ghcr.io/immich-app/immich-web:release
entrypoint: [ "/bin/sh", "./entrypoint.sh" ]
env_file:
- .env
restart: always
typesense:
container_name: immich_typesense
image: typesense/typesense:0.24.0
environment:
- TYPESENSE_API_KEY=${TYPESENSE_API_KEY}
- TYPESENSE_DATA_DIR=/data
logging:
driver: none
volumes:
- tsdata:/data
restart: always
redis:
container_name: immich_redis
image: redis:6.2
restart: always
database:
container_name: immich_postgres
image: postgres:14
env_file:
- .env
environment:
POSTGRES_PASSWORD: ${DB_PASSWORD}
POSTGRES_USER: ${DB_USERNAME}
POSTGRES_DB: ${DB_DATABASE_NAME}
PG_DATA: /var/lib/postgresql/data
volumes:
- pgdata:/var/lib/postgresql/data
restart: always
immich-proxy:
container_name: immich_proxy
image: ghcr.io/immich-app/immich-proxy:release
environment:
# Make sure these values get passed through from the env file
- IMMICH_SERVER_URL
- IMMICH_WEB_URL
ports:
- 2283:8080
logging:
driver: none
depends_on:
- immich-server
restart: always
volumes:
pgdata:
model-cache:
tsdata:
Your .env content
.env content
```Shell ################################################################################### # Database ###################################################################################DB_HOSTNAME=immich_postgres DB_USERNAME=postgres DB_PASSWORD=postgres DB_DATABASE_NAME=immich
Optional Database settings:
DB_PORT=5432
###################################################################################
Redis
###################################################################################
REDIS_HOSTNAME=immich_redis
Optional Redis settings:
Note: these parameters are not automatically passed to the Redis Container
to do so, please edit the docker-compose.yml file as well. Redis is not configured
via environment variables, only redis.conf or the command line
REDIS_PORT=6379
REDIS_DBINDEX=0
REDIS_PASSWORD=
REDIS_SOCKET=
###################################################################################
Upload File Location
This is the location where uploaded files are stored.
###################################################################################
UPLOAD_LOCATION=/mnt/disk1/im_photos/
###################################################################################
Typesense
################################################################################### TYPESENSE_API_KEY=sdfuysdfuywnrkvhvslkddgsb #TYPESENSE_ENABLED=false
###################################################################################
Reverse Geocoding
Reverse geocoding is done locally which has a small impact on memory usage
This memory usage can be altered by changing the REVERSE_GEOCODING_PRECISION variable
This ranges from 0-3 with 3 being the most precise
3 - Cities > 500 population: ~200MB RAM
2 - Cities > 1000 population: ~150MB RAM
1 - Cities > 5000 population: ~80MB RAM
0 - Cities > 15000 population: ~40MB RAM
####################################################################################
DISABLE_REVERSE_GEOCODING=false
REVERSE_GEOCODING_PRECISION=3
####################################################################################
WEB - Optional
Custom message on the login page, should be written in HTML form.
For example:
PUBLIC_LOGIN_PAGE_MESSAGE="This is a demo instance of Immich.
Email: [email protected]
Password: demo"
####################################################################################
PUBLIC_LOGIN_PAGE_MESSAGE=
####################################################################################
Alternative Service Addresses - Optional
This is an advanced feature for users who may be running their immich services on different hosts.
It will not change which address or port that services bind to within their containers, but it will change where other services look for their peers.
Note: immich-microservices is bound to 3002, but no references are made
####################################################################################
IMMICH_WEB_URL=http://immich-web:3000 IMMICH_SERVER_URL=http://immich-server:3001 IMMICH_MACHINE_LEARNING_URL=http://immich-machine-learning:3003
####################################################################################
Alternative API's External Address - Optional
This is an advanced feature used to control the public server endpoint returned to clients during Well-known discovery.
You should only use this if you want mobile apps to access the immich API over a custom URL. Do not include trailing slash.
NOTE: At this time, the web app will not be affected by this setting and will continue to use the relative path: /api
Examples: http://localhost:3001, http://immich-api.example.com, etc
####################################################################################
#IMMICH_API_URL_EXTERNAL=http://localhost:3001
</details>
### Reproduction steps
```bash
Install the latest version, check "Explore" in the web version and things search in the mobile version.
Additional information
The database has over 169,000 photos and 2,000 videos.
Can you try removing the tsdata volume and restarting the stack? It seems to have an issue during indexing to show that error
Can you try removing the tsdata volume and restarting the stack? It seems to have an issue during indexing to show that error
I delete the volume, after restarting the same error.
This is pretty strange, can you bring down the whole stack, remove typesense container and typesense volume and bring them up? It would be helpful if you also capture screenshot along the way
This is pretty strange, can you bring down the whole stack, remove typesense container and typesense volume and bring them up? It would be helpful if you also capture screenshot along the way
Ok, if I understand you correctly:
Ok, i down full stack

now i remove TS volume


There are no Immich containers

Now we pull and up it

And we have some error


I think indexing is going on in the background (microservices container). Can you share those logs?
Got that issue last week but is expected as I did TYPESENSE_ENABLED = false and I've uploaded more pictures via the web. Fixed that after I did an Encode Clip job run after I reverted back to TYPESENSE_ENABLED = true;.
Try to run the Encode Clip job again on missing items.
I think indexing is going on in the background (microservices container). Can you share those logs?
Yes, found some error in microservices log
microservices log
Processor] [39m[32mInitializing Reverse Geocoding[39m /usr/src/app/node_modules/local-reverse-geocoder/index.js:746 throw err; ^CsvError: Invalid Record Length: expect 19, got 18 on line 60031 at Object.__onRecord (/usr/src/app/node_modules/csv-parse/dist/cjs/index.cjs:940:11) at Object.parse (/usr/src/app/node_modules/csv-parse/dist/cjs/index.cjs:903:36) at Parser._flush (/usr/src/app/node_modules/csv-parse/dist/cjs/index.cjs:1336:26) at Parser.final [as _final] (node:internal/streams/transform:112:25) at callFinal (node:internal/streams/writable:694:27) at prefinish (node:internal/streams/writable:723:7) at finishMaybe (node:internal/streams/writable:733:5) at afterWrite (node:internal/streams/writable:504:3) at onwrite (node:internal/streams/writable:477:7) at Parser.Transform._read (node:internal/streams/transform:245:5) { code: 'CSV_RECORD_INCONSISTENT_FIELDS_LENGTH', bytes: 10502144, comment_lines: 0, empty_lines: 0, invalid_field_length: 0, lines: 60031, records: 60030, columns: false, error: undefined, header: false, index: 18, raw: undefined, column: 18, quoting: false, record: [ '2965092', 'Courtown', 'Courtown', 'Baile na Cuirte,Baile na Cúirte,Courtown,Courtown Harbour,Cuan Bhaile na Cuirte,Cuan Bhaile na Cúirte', '52.64424', '-6.22899', 'P', 'PPL', 'IE', '', 'L', '30', '', '', '3591', '', '9', '' ] } [32m[Nest] 1 - [39m04/26/2023, 12:00:19 AM [32m LOG[39m [38;5;3m[NestFactory] [39m[32mStarting Nest application...[39m [32m[Nest] 1 - [39m04/26/2023, 12:00:19 AM [32m LOG[39m [38;5;3m[InstanceLoader] [39m[32mTypeOrmModule dependencies initialized[39m[38;5;3m +83ms[39m [32m[Nest] 1 - [39m04/26/2023, 12:00:19 AM [32m LOG[39m [38;5;3m[InstanceLoader] [39m[32mBullModule dependencies initialized[39m[38;5;3m +2ms[39m [32m[Nest] 1 - [39m04/26/2023, 12:00:19 AM [32m LOG[39m [38;5;3m[InstanceLoader] [39m[32mConfigHostModule dependencies initialized[39m[38;5;3m +1ms[39m [32m[Nest] 1 - [39m04/26/2023, 12:00:19 AM [32m LOG[39m [38;5;3m[InstanceLoader] [39m[32mDiscoveryModule dependencies initialized[39m[38;5;3m +1ms[39m [32m[Nest] 1 - [39m04/26/2023, 12:00:19 AM [32m LOG[39m [38;5;3m[InstanceLoader] [39m[32mConfigModule dependencies initialized[39m[38;5;3m +12ms[39m [32m[Nest] 1 - [39m04/26/2023, 12:00:19 AM [32m LOG[39m [38;5;3m[InstanceLoader] [39m[32mBullModule dependencies initialized[39m[38;5;3m +1ms[39m [32m[Nest] 1 - [39m04/26/2023, 12:00:19 AM [32m LOG[39m [38;5;3m[InstanceLoader] [39m[32mBullModule dependencies initialized[39m[38;5;3m +1ms[39m [32m[Nest] 1 - [39m04/26/2023, 12:00:19 AM [32m LOG[39m [38;5;3m[InstanceLoader] [39m[32mTypeOrmCoreModule dependencies initialized[39m[38;5;3m +217ms[39m [32m[Nest] 1 - [39m04/26/2023, 12:00:19 AM [32m LOG[39m [38;5;3m[InstanceLoader] [39m[32mTypeOrmModule dependencies initialized[39m[38;5;3m +0ms[39m [32m[Nest] 1 - [39m04/26/2023, 12:00:19 AM [32m LOG[39m [38;5;3m[InstanceLoader] [39m[32mTypeOrmModule dependencies initialized[39m[38;5;3m +0ms[39m [33m[Nest] 1 - [39m04/26/2023, 12:00:19 AM [33m WARN[39m [38;5;3m[MetadataExtractionProcessor] [39m[33mReverse geocoding is enabled[39m [32m[Nest] 1 - [39m04/26/2023, 12:00:19 AM [32m LOG[39m [38;5;3m[MetadataExtractionProcessor] [39m[32mInitializing Reverse Geocoding[39m [32m[Nest] 1 - [39m04/26/2023, 12:00:19 AM [32m LOG[39m [38;5;3m[InstanceLoader] [39m[32mInfraModule dependencies initialized[39m[38;5;3m +36ms[39m [32m[Nest] 1 - [39m04/26/2023, 12:00:20 AM [32m LOG[39m [38;5;3m[InstanceLoader] [39m[32mDomainModule dependencies initialized[39m[38;5;3m +1ms[39m [32m[Nest] 1 - [39m04/26/2023, 12:00:20 AM [32m LOG[39m [38;5;3m[InstanceLoader] [39m[32mMicroservicesModule dependencies initialized[39m[38;5;3m +0ms[39m [32m[Nest] 1 - [39m04/26/2023, 12:00:20 AM [32m LOG[39m [38;5;3m[NestApplication] [39m[32mNest application successfully started[39m[38;5;3m +24ms[39m [32m[Nest] 1 - [39m04/26/2023, 12:00:20 AM [32m LOG[39m [38;5;3m[ImmichMicroservice] [39m[32mRunning Immich Microservices in PRODUCTION environment - version 1.54.1 - Listening on port: 3002[39m [32m[Nest] 1 - [39m04/26/2023, 12:00:32 AM [32m LOG[39m [38;5;3m[MetadataExtractionProcessor] [39m[32mReverse Geocoding Initialized[39m
<--- Last few GCs --->
[1:0x7f0f65ef1330] 128394 ms: Scavenge (reduce) 2046.4 (2083.0) -> 2045.5 (2083.2) MB, 13.3 / 0.0 ms (average mu = 0.239, current mu = 0.173) allocation failure [1:0x7f0f65ef1330] 128903 ms: Mark-sweep (reduce) 2046.4 (2083.2) -> 2041.9 (2083.5) MB, 438.2 / 0.0 ms (+ 2285.4 ms in 592 steps since start of marking, biggest step 33.5 ms, walltime since start of marking 3380 ms) (average mu = 0.235, current mu = 0
<--- JS stacktrace --->
FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory [32m[Nest] 1 - [39m04/26/2023, 12:02:33 AM [32m LOG[39m [38;5;3m[NestFactory] [39m[32mStarting Nest application...[39m [32m[Nest] 1 - [39m04/26/2023, 12:02:33 AM [32m LOG[39m [38;5;3m[InstanceLoader] [39m[32mTypeOrmModule dependencies initialized[39m[38;5;3m +77ms[39m [32m[Nest] 1 - [39m04/26/2023, 12:02:33 AM [32m LOG[39m [38;5;3m[InstanceLoader] [39m[32mBullModule dependencies initialized[39m[38;5;3m +0ms[39m [32m[Nest] 1 - [39m04/26/2023, 12:02:33 AM [32m LOG[39m [38;5;3m[InstanceLoader] [39m[32mConfigHostModule dependencies initialized[39m[38;5;3m +1ms[39m [32m[Nest] 1 - [39m04/26/2023, 12:02:33 AM [32m LOG[39m [38;5;3m[InstanceLoader] [39m[32mDiscoveryModule dependencies initialized[39m[38;5;3m +1ms[39m [32m[Nest] 1 - [39m04/26/2023, 12:02:33 AM [32m LOG[39m [38;5;3m[InstanceLoader] [39m[32mConfigModule dependencies initialized[39m[38;5;3m +10ms[39m [32m[Nest] 1 - [39m04/26/2023, 12:02:33 AM [32m LOG[39m [38;5;3m[InstanceLoader] [39m[32mBullModule dependencies initialized[39m[38;5;3m +0ms[39m [32m[Nest] 1 - [39m04/26/2023, 12:02:33 AM [32m LOG[39m [38;5;3m[InstanceLoader] [39m[32mBullModule dependencies initialized[39m[38;5;3m +1ms[39m [32m[Nest] 1 - [39m04/26/2023, 12:02:33 AM [32m LOG[39m [38;5;3m[InstanceLoader] [39m[32mTypeOrmCoreModule dependencies initialized[39m[38;5;3m +205ms[39m [32m[Nest] 1 - [39m04/26/2023, 12:02:33 AM [32m LOG[39m [38;5;3m[InstanceLoader] [39m[32mTypeOrmModule dependencies initialized[39m[38;5;3m +1ms[39m [32m[Nest] 1 - [39m04/26/2023, 12:02:33 AM [32m LOG[39m [38;5;3m[InstanceLoader] [39m[32mTypeOrmModule dependencies initialized[39m[38;5;3m +0ms[39m [33m[Nest] 1 - [39m04/26/2023, 12:02:33 AM [33m WARN[39m [38;5;3m[MetadataExtractionProcessor] [39m[33mReverse geocoding is enabled[39m [32m[Nest] 1 - [39m04/26/2023, 12:02:33 AM [32m LOG[39m [38;5;3m[MetadataExtractionProcessor] [39m[32mInitializing Reverse Geocoding[39m [32m[Nest] 1 - [39m04/26/2023, 12:02:43 AM [32m LOG[39m [38;5;3m[InstanceLoader] [39m[32mInfraModule dependencies initialized[39m[38;5;3m +9424ms[39m [32m[Nest] 1 - [39m04/26/2023, 12:02:43 AM [32m LOG[39m [38;5;3m[InstanceLoader] [39m[32mDomainModule dependencies initialized[39m[38;5;3m +1ms[39m [32m[Nest] 1 - [39m04/26/2023, 12:02:43 AM [32m LOG[39m [38;5;3m[InstanceLoader] [39m[32mMicroservicesModule dependencies initialized[39m[38;5;3m +1ms[39m [32m[Nest] 1 - [39m04/26/2023, 12:02:43 AM [32m LOG[39m [38;5;3m[NestApplication] [39m[32mNest application successfully started[39m[38;5;3m +23ms[39m [32m[Nest] 1 - [39m04/26/2023, 12:02:43 AM [32m LOG[39m [38;5;3m[ImmichMicroservice] [39m[32mRunning Immich Microservices in PRODUCTION environment - version 1.54.1 - Listening on port: 3002[39m [32m[Nest] 1 - [39m04/26/2023, 12:02:43 AM [32m LOG[39m [38;5;3m[MetadataExtractionProcessor] [39m[32mReverse Geocoding Initialized[39m [32m[Nest] 1 - [39m04/26/2023, 8:46:55 AM [32m LOG[39m [38;5;3m[NestFactory] [39m[32mStarting Nest application...[39m [32m[Nest] 1 - [39m04/26/2023, 8:46:55 AM [32m LOG[39m [38;5;3m[InstanceLoader] [39m[32mTypeOrmModule dependencies initialized[39m[38;5;3m +78ms[39m [32m[Nest] 1 - [39m04/26/2023, 8:46:55 AM [32m LOG[39m [38;5;3m[InstanceLoader] [39m[32mBullModule dependencies initialized[39m[38;5;3m +1ms[39m [32m[Nest] 1 - [39m04/26/2023, 8:46:55 AM [32m LOG[39m [38;5;3m[InstanceLoader] [39m[32mConfigHostModule dependencies initialized[39m[38;5;3m +1ms[39m [32m[Nest] 1 - [39m04/26/2023, 8:46:55 AM [32m LOG[39m [38;5;3m[InstanceLoader] [39m[32mDiscoveryModule dependencies initialized[39m[38;5;3m +1ms[39m [32m[Nest] 1 - [39m04/26/2023, 8:46:55 AM [32m LOG[39m [38;5;3m[InstanceLoader] [39m[32mConfigModule dependencies initialized[39m[38;5;3m +9ms[39m [32m[Nest] 1 - [39m04/26/2023, 8:46:55 AM [32m LOG[39m [38;5;3m[InstanceLoader] [39m[32mBullModule dependencies initialized[39m[38;5;3m +1ms[39m [32m[Nest] 1 - [39m04/26/2023, 8:46:55 AM [32m LOG[39m [38;5;3m[InstanceLoader] [39m[32mBullModule dependencies initialized[39m[38;5;3m +1ms[39m [32m[Nest] 1 - [39m04/26/2023, 8:46:56 AM [32m LOG[39m [38;5;3m[InstanceLoader] [39m[32mTypeOrmCoreModule dependencies initialized[39m[38;5;3m +240ms[39m [32m[Nest] 1 - [39m04/26/2023, 8:46:56 AM [32m LOG[39m [38;5;3m[InstanceLoader] [39m[32mTypeOrmModule dependencies initialized[39m[38;5;3m +1ms[39m [32m[Nest] 1 - [39m04/26/2023, 8:46:56 AM [32m LOG[39m [38;5;3m[InstanceLoader] [39m[32mTypeOrmModule dependencies initialized[39m[38;5;3m +0ms[39m [33m[Nest] 1 - [39m04/26/2023, 8:46:56 AM [33m WARN[39m [38;5;3m[MetadataExtractionProcessor] [39m[33mReverse geocoding is enabled[39m [32m[Nest] 1 - [39m04/26/2023, 8:46:56 AM [32m LOG[39m [38;5;3m[MetadataExtractionProcessor] [39m[32mInitializing Reverse Geocoding[39m [32m[Nest] 1 - [39m04/26/2023, 8:46:56 AM [32m LOG[39m [38;5;3m[InstanceLoader] [39m[32mInfraModule dependencies initialized[39m[38;5;3m +9ms[39m [32m[Nest] 1 - [39m04/26/2023, 8:46:56 AM [32m LOG[39m [38;5;3m[InstanceLoader] [39m[32mDomainModule dependencies initialized[39m[38;5;3m +1ms[39m [32m[Nest] 1 - [39m04/26/2023, 8:46:56 AM [32m LOG[39m [38;5;3m[InstanceLoader] [39m[32mMicroservicesModule dependencies initialized[39m[38;5;3m +0ms[39m [32m[Nest] 1 - [39m04/26/2023, 8:46:56 AM [32m LOG[39m [38;5;3m[NestApplication] [39m[32mNest application successfully started[39m[38;5;3m +25ms[39m [32m[Nest] 1 - [39m04/26/2023, 8:46:56 AM [32m LOG[39m [38;5;3m[ImmichMicroservice] [39m[32mRunning Immich Microservices in PRODUCTION environment - version 1.54.1 - Listening on port: 3002[39m
Got that issue last week but is expected as I did TYPESENSE_ENABLED = false and I've uploaded more pictures via the web. Fixed that after I did an Encode Clip job run after I reverted back to TYPESENSE_ENABLED = true;.
Try to run the Encode Clip job again on missing items.
I tried this and it didn't work.
Your container is in an invalid state due to another bug. Delete and recreate the container and it should start up. Then wait for indexing to finish.