immich Uploading large video file via immich cli fails

The bug

When I try to upload a ~11GB video file via immich cli on windows I get the following error:

file:///C:/Users/user/scoop/persist/nodejs/bin/node_modules/@immich/cli/dist/index.js:9719
            throw new FetchError(e, "The request failed and the interceptors did not return an alternative response");
                  ^

FetchError: The request failed and the interceptors did not return an alternative response
    at AssetApi.fetchApi (file:///C:/Users/user/scoop/persist/nodejs/bin/node_modules/@immich/cli/dist/index.js:9719:19)
    at async AssetApi.request (file:///C:/Users/user/scoop/persist/nodejs/bin/node_modules/@immich/cli/dist/index.js:9771:22)
    ... 3 lines matching cause stack trace ...
    at async Command.<anonymous> (file:///C:/Users/user/scoop/persist/nodejs/bin/node_modules/@immich/cli/dist/index.js:22454:3) {
  cause: TypeError: fetch failed
      at node:internal/deps/undici/undici:12442:11
      at async AssetApi.fetchApi (file:///C:/Users/user/scoop/persist/nodejs/bin/node_modules/@immich/cli/dist/index.js:9704:20)
      at async AssetApi.request (file:///C:/Users/user/scoop/persist/nodejs/bin/node_modules/@immich/cli/dist/index.js:9771:22)
      at async AssetApi.uploadFileRaw (file:///C:/Users/user/scoop/persist/nodejs/bin/node_modules/@immich/cli/dist/index.js:13469:22)
      at async AssetApi.uploadFile (file:///C:/Users/user/scoop/persist/nodejs/bin/node_modules/@immich/cli/dist/index.js:13481:22)
      at async UploadCommand.run (file:///C:/Users/user/scoop/persist/nodejs/bin/node_modules/@immich/cli/dist/index.js:22389:30)
      at async Command.<anonymous> (file:///C:/Users/user/scoop/persist/nodejs/bin/node_modules/@immich/cli/dist/index.js:22454:3) {
    cause: RequestContentLengthMismatchError: Request body length does not match content-length header
        at AsyncWriter.write (node:internal/deps/undici/undici:9780:19)
        at writeIterable (node:internal/deps/undici/undici:9740:23) {
      code: 'UND_ERR_REQ_CONTENT_LENGTH_MISMATCH'
    }
  }
}

I connect directly to the immich docker ocntainer. There is no error message in the immich docker logs.

The OS that Immich Server is running on

Docker for Windows on Windows 11

Version of Immich Server

v1.94.1

Version of Immich Mobile App

v1.94.1

Platform with the issue

[X] Server
[ ] Web
[ ] Mobile

Your docker-compose.yml content

immich-server:
    container_name: immich-server
    image: ghcr.io/immich-app/immich-server:release
    command: [ "start.sh", "immich" ]
    networks:
      t2_proxy:
        ipv4_address: 192.168.90.88
    volumes:
      - d:/bilder/immich:/usr/src/app/upload
    ports:
      - "13001:3001"
    env_file:
      - ../immich/immich.env
    environment:
      - NODE_ENV=production
    depends_on:
      - immich-redis
      - immich-postgres
    restart: unless-stopped
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.immich-rtr.entrypoints=https"
      - "traefik.http.routers.immich-rtr.rule=Host(`example.com`)" 
      - "traefik.http.routers.immich-rtr.service=immich-svc"
      - "traefik.http.services.immich-svc.loadbalancer.server.port=3001"

  immich-microservices:
    container_name: immich-microservices
    image: ghcr.io/immich-app/immich-server:release
    command:  [ "start.sh", "microservices" ]
    networks:
      t2_proxy:
        ipv4_address: 192.168.90.89
    volumes:
      - d:/bilder/immich:/usr/src/app/upload
    env_file:
      - ../immich/immich.env
    environment:
      - NODE_ENV=production
    depends_on:
      - immich-redis
      - immich-postgres
    restart: unless-stopped

  immich-machine-learning:
    container_name: immich-machine-learning
    image: ghcr.io/immich-app/immich-machine-learning:release
    networks:
      t2_proxy:
        ipv4_address: 192.168.90.90
    volumes:
      - c:/docker/immich/cache:/cache
    env_file:
      - ../immich/immich.env
    environment:
      - NODE_ENV=production
    restart: unless-stopped

  immich-redis:
    container_name: immich-redis
    image: redis:6.2-alpine@sha256:70a7a5b641117670beae0d80658430853896b5ef269ccf00d1827427e3263fa3
    networks:
      t2_proxy:
        ipv4_address: 192.168.90.92
    restart: unless-stopped
    
  immich-postgres:
    container_name: immich-postgres
    image: tensorchord/pgvecto-rs:pg14-v0.1.11
    networks:
      t2_proxy:
        ipv4_address: 192.168.90.93
    env_file:
      - ../immich/immich.env
    environment:
      POSTGRES_PASSWORD: xxx
      POSTGRES_USER: postgres
      POSTGRES_DB: immich
      PG_DATA: /var/lib/postgresql/data
    volumes:
      - immich_db_data:/var/lib/postgresql/data
    restart: unless-stopped

Your .env content

###################################################################################
# Database
###################################################################################

DB_HOSTNAME=immich-postgres
DB_USERNAME=postgres
DB_PASSWORD=xxx
DB_DATABASE_NAME=immich

# Optional Database settings:
# DB_PORT=5432

###################################################################################
# Redis
###################################################################################

REDIS_HOSTNAME=immich-redis

# Optional Redis settings:
# REDIS_PORT=6379
# REDIS_DBINDEX=0
# REDIS_PASSWORD=
# REDIS_SOCKET=

###################################################################################
# Upload File Location
#
# This is the location where uploaded files are stored.
###################################################################################

UPLOAD_LOCATION=d:/bilder/immich/

###################################################################################
# JWT SECRET
#
# This JWT_SECRET is used to sign the authentication keys for user login
# You should set it to a long randomly generated value
# You can use this command to generate one: openssl rand -base64 128
###################################################################################

JWT_SECRET=xxx

###################################################################################
# Reverse Geocoding
#
# Reverse geocoding is done locally which has a small impact on memory usage
# This memory usage can be altered by changing the REVERSE_GEOCODING_PRECISION variable
# This ranges from 0-3 with 3 being the most precise
# 3 - Cities > 500 population: ~200MB RAM
# 2 - Cities > 1000 population: ~150MB RAM
# 1 - Cities > 5000 population: ~80MB RAM
# 0 - Cities > 15000 population: ~40MB RAM
####################################################################################

# DISABLE_REVERSE_GEOCODING=false
# REVERSE_GEOCODING_PRECISION=3

####################################################################################
# WEB - Optional
#
# Custom message on the login page, should be written in HTML form.
# For example:
# PUBLIC_LOGIN_PAGE_MESSAGE="This is a demo instance of Immich.<br><br>Email: <i>[email protected]</i><br>Password: <i>demo</i>"
####################################################################################

PUBLIC_LOGIN_PAGE_MESSAGE=

####################################################################################
# Alternative Service Addresses - Optional
#
# This is an advanced feature for users who may be running their immich services on different hosts.
# It will not change which address or port that services bind to within their containers, but it will change where other services look for their peers.
# Note: immich-microservices is bound to 3002, but no references are made
####################################################################################

IMMICH_WEB_URL=http://immich-web:3000
IMMICH_SERVER_URL=http://immich-server:3001
IMMICH_MACHINE_LEARNING_URL=http://immich-machine-learning:3003

Reproduction steps

Upload a big video file via immich cli

Additional information

No response

Feb 12 '24 17:02 Froghut

I just ran into the same problem yesterday when trying to upload video files that are greater than 4GB. I'm running v1.94.1 on the server. I used the docker method to run the cli import tool and am using version latest on that container. Finding these large files find . -type f -size +4G and moving them outside of the import location seems to help but would be best if we could import these large files. The files that are causing this error on are .MOV files from my iPhone.

Feb 13 '24 16:02 lakestonelabs

I also just received this error when attempting to upload a large collection of images and videos through the CLI:

docker run -it -v "$(pwd)":/import:ro -e IMMICH_INSTANCE_URL=http://192.168.0.3:8082/api -e IMMICH_API_KEY=KEY_WAS_HERE -v '/mnt/user/family/Photos':'/photo-import':'rw' ghcr.io/immich-app/immich-cli:latest upload --include-hidden --recursive /photo-import

file:///usr/src/app/dist/index.js:9719
            throw new FetchError(e, "The request failed and the interceptors did not return an alternative response");
                  ^

FetchError: The request failed and the interceptors did not return an alternative response
    at AssetApi.fetchApi (file:///usr/src/app/dist/index.js:9719:19)
    at async AssetApi.request (file:///usr/src/app/dist/index.js:9771:22)
    ... 3 lines matching cause stack trace ...
    at async Command.<anonymous> (file:///usr/src/app/dist/index.js:22125:3) {
  cause: TypeError: fetch failed
      at node:internal/deps/undici/undici:12344:11
      at async AssetApi.fetchApi (file:///usr/src/app/dist/index.js:9704:20)
      at async AssetApi.request (file:///usr/src/app/dist/index.js:9771:22)
      at async AssetApi.uploadFileRaw (file:///usr/src/app/dist/index.js:13168:22)
      at async AssetApi.uploadFile (file:///usr/src/app/dist/index.js:13180:22)
      at async UploadCommand.run (file:///usr/src/app/dist/index.js:22060:30)
      at async Command.<anonymous> (file:///usr/src/app/dist/index.js:22125:3) {
    cause: RequestContentLengthMismatchError: Request body length does not match content-length header
        at AsyncWriter.write (node:internal/deps/undici/undici:9682:19)
        at writeIterable (node:internal/deps/undici/undici:9642:23) {
      code: 'UND_ERR_REQ_CONTENT_LENGTH_MISMATCH'
    }
  }
}

The file that it tripped on is a 11.7 GB large MP4. When I reran the import the script halted on the same file a second time. This is on an Unraid system with 128 GB of RAM.

For a work-around I'll attempt to remove anything larger than 4GB from the import directory like @lakestonelabs suggested.

If importing files this large isn't a possibility, we should at least be catching the error, logging it, and continuing with the import.

Feb 20 '24 00:02 refringe

Same problem when using deployment via Docker Compose on Ubuntu 22.04. Cli is installed on Node v21.6.2 on same machine

Feb 27 '24 04:02 justin13888

Also, same files managed to upload over website so it's more likely an issue with CLI

Feb 27 '24 07:02 justin13888

Same behaviour, size bigger than 4Go failed to import. And the --ignore parameters does not seem to work, I had to move the files. Also, when I upload a lot of files, I have to disable all jobs, otherwise the uploads fails.

Version : v1.97.0

Mar 07 '24 10:03 Zelnes

I am experiencing this same issue running Immich 1.98.1 on a Compose setup with Node.js v20.11.1 and Immich CLI 2.0.7.

$ docker exec -ti immich_server bash
node@207a6cbdc17a:/usr/src/app$ id -u
1000
node@207a6cbdc17a:/usr/src/app$ cd upload/User
node@207a6cbdc17a:/usr/src/app/upload/User$ ls -lhAF
-rw-rw-rw- 1 99 users 5.6G Mar 10  2024  big_file.mp4
node@207a6cbdc17a:/usr/src/app/upload/User$ immich login-key http://localhost:3001/api MY_API_KEY
Logging in...
Logged in as [email protected]
Wrote auth info to /home/node/.config/immich/auth.yml
node@207a6cbdc17a:/usr/src/app/upload/User$ time immich upload /usr/src/app/upload/User
░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ | 0% | ETA: 0s | 0/5.6 GB: /usr/src/app/upload/User/big_file.mp4
file:///usr/src/app/node_modules/@immich/cli/dist/index.js:9719
            throw new FetchError(e, "The request failed and the interceptors did not return an alternative response");
                  ^

FetchError: The request failed and the interceptors did not return an alternative response
    at AssetApi.fetchApi (file:///usr/src/app/node_modules/@immich/cli/dist/index.js:9719:19)
    at async AssetApi.request (file:///usr/src/app/node_modules/@immich/cli/dist/index.js:9771:22)
    ... 3 lines matching cause stack trace ...
    at async Command.<anonymous> (file:///usr/src/app/node_modules/@immich/cli/dist/index.js:22454:3) {
  cause: TypeError: fetch failed
      at node:internal/deps/undici/undici:12345:11
      at async AssetApi.fetchApi (file:///usr/src/app/node_modules/@immich/cli/dist/index.js:9704:20)
      at async AssetApi.request (file:///usr/src/app/node_modules/@immich/cli/dist/index.js:9771:22)
      at async AssetApi.uploadFileRaw (file:///usr/src/app/node_modules/@immich/cli/dist/index.js:13469:22)
      at async AssetApi.uploadFile (file:///usr/src/app/node_modules/@immich/cli/dist/index.js:13481:22)
      at async UploadCommand.run (file:///usr/src/app/node_modules/@immich/cli/dist/index.js:22389:30)
      at async Command.<anonymous> (file:///usr/src/app/node_modules/@immich/cli/dist/index.js:22454:3) {
    cause: RequestContentLengthMismatchError: Request body length does not match content-length header
        at AsyncWriter.write (node:internal/deps/undici/undici:9682:19)
        at writeIterable (node:internal/deps/undici/undici:9642:23) {
      code: 'UND_ERR_REQ_CONTENT_LENGTH_MISMATCH'
    }
  }
}

Mar 11 '24 02:03 maximberezin97

Affects CLI v2.1.0 as well, but gives a slightly different error message

$ immich upload IMG_5820.MOV
Crawling for assets...
Checking assets | ████████████████████████████████████████ | 100% | ETA: 0s | 2/2 assets
Uploading assets | ░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ | 0% | ETA: 0s | 0/21.0 GB
node:internal/deps/undici/undici:13737
      Error.captureStackTrace(err, this);
            ^

TypeError: fetch failed
    at node:internal/deps/undici/undici:13737:13
    at async UploadCommand.uploadAsset (file:///home/datatags/.nvm/versions/node/v21.7.1/lib/node_modules/@immich/cli/dist/index.js:18473:22)
    at async Promise.all (index 0)
    at async UploadCommand.upload (file:///home/datatags/.nvm/versions/node/v21.7.1/lib/node_modules/@immich/cli/dist/index.js:18297:21)
    at async UploadCommand.run (file:///home/datatags/.nvm/versions/node/v21.7.1/lib/node_modules/@immich/cli/dist/index.js:18220:31)
    at async Command.<anonymous> (file:///home/datatags/.nvm/versions/node/v21.7.1/lib/node_modules/@immich/cli/dist/index.js:18501:3) {
  [cause]: RequestContentLengthMismatchError: Request body length does not match content-length header
      at AsyncWriter.write (node:internal/deps/undici/undici:10497:19)
      at writeIterable (node:internal/deps/undici/undici:10457:23) {
    code: 'UND_ERR_REQ_CONTENT_LENGTH_MISMATCH'
  }
}

Node.js v21.7.1

Mar 12 '24 17:03 datatags

I wonder if its a greater issue than just CLI. I am unable to upload files larger than 150MB through the browser right now, but if I upload through the mobile app it works just fine. Mobile app I tested to upload 6GB files.

Mar 12 '24 18:03 alternativesurfer

This is definitely still an issue

Mar 16 '24 08:03 mccarverone

And it affects 1.99.0 as well.

Mar 20 '24 20:03 mccarverone

immich immich copied to clipboard

Uploading large video file via immich cli fails

The bug

The OS that Immich Server is running on

Version of Immich Server

Version of Immich Mobile App

Platform with the issue

Your docker-compose.yml content

Your .env content

Reproduction steps

Additional information

immich
immich copied to clipboard