whishper icon indicating copy to clipboard operation
whishper copied to clipboard

Published 20 hours ago • verified publisher icon pluja

Unsupported compute type using GPU

Open maccaveli opened this issue 8 months ago • 1 comments

Description

Transcription fails using GPU support on .mp4 file with Nvidia GTX 980 because of unsupported compute type. Is there a way to manually change the compute type?

Error: ValueError: Requested int8 compute type, but the target device or backend do not support efficient int8 computation.

To Reproduce

Enable GPU support according to documentation (https://whishper.net/guides/gpu/) by replacing docker-yml file, running "docker-compose up -d" and transcribing .mp4 using GPU.

Expected behavior

Successful transcription using GPU. Using CPU works perfectly well.

Environment

  • OS: Windows 10 Pro x64
  • Docker 4.25.0 (126437)
  • Nvidia 536.23 driver

Logs and Configuration

Docker Compose Logs

Run the following command in the project folder, force the error, and paste the logs below: docker compose logs -f --tail 50

whishper-mongo-1        | {"t":{"$date":"2023-11-02T07:15:42.349Z"},"s":"I",  "c":"CONTROL",  "id":20697,   "ctx":"main","msg":"Renamed existing log file","attr":{"oldLogPath":"/var/log/mongodb/mongod.log","newLogPath":"/var/log/mongodb/mongod.log.2023-11-02T07-15-42"}}
whishper-nginx-1        | /docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
whishper-nginx-1        | /docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
whishper-nginx-1        | /docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
whishper-nginx-1        | 10-listen-on-ipv6-by-default.sh: info: Getting the checksum of /etc/nginx/conf.d/default.conf
whishper-nginx-1        | 10-listen-on-ipv6-by-default.sh: info: Enabled listen on IPv6 in /etc/nginx/conf.d/default.conf
whishper-nginx-1        | /docker-entrypoint.sh: Sourcing /docker-entrypoint.d/15-local-resolvers.envsh
whishper-nginx-1        | /docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh
whishper-nginx-1        | /docker-entrypoint.sh: Launching /docker-entrypoint.d/30-tune-worker-processes.sh
whishper-nginx-1        | /docker-entrypoint.sh: Configuration complete; ready for start up
whishper-nginx-1        | 172.18.0.2 - - [02/Nov/2023:07:17:24 +0000] "GET /api/transcriptions HTTP/1.1" 200 1815534 "-" "undici"
whishper-nginx-1        | 172.18.0.1 - - [02/Nov/2023:07:17:24 +0000] "GET / HTTP/1.1" 200 515561 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36"
whishper-backend        | 7:17AM INF Starting monitor!
whishper-backend        |
whishper-backend        |  ┌───────────────────────────────────────────────────┐
whishper-backend        |  │                   Fiber v2.50.0                   │
whishper-backend        |  │               http://127.0.0.1:8080               │
whishper-backend        |  │       (bound on host 0.0.0.0 and port 8080)       │
whishper-backend        |  │                                                   │
whishper-backend        |  │ Handlers ............ 13  Processes ........... 1 │
whishper-backend        |  │ Prefork ....... Disabled  PID ................. 1 │
whishper-backend        |  └───────────────────────────────────────────────────┘
whishper-backend        |
whishper-backend        | 7:17AM ERR Error sending transcription request error="invalid status"
whishper-backend        | 7:17AM ERR Error transcribing error="invalid status"
whishper-backend        | 7:18AM ERR Error reading message error="websocket: close 1006 (abnormal closure): unexpected EOF"
whishper-backend        | 7:19AM ERR Error reading message error="websocket: close 1006 (abnormal closure): unexpected EOF"
whishper-backend        | 7:20AM ERR Error reading message error="websocket: close 1006 (abnormal closure): unexpected EOF"
whishper-backend        | 7:21AM ERR Error reading message error="websocket: close 1006 (abnormal closure): unexpected EOF"
whishper-backend        | 7:22AM ERR Error reading message error="websocket: close 1006 (abnormal closure): unexpected EOF"
whishper-backend        | 7:24AM ERR Error reading message error="websocket: close 1006 (abnormal closure): unexpected EOF"
whishper-backend        | 7:25AM ERR Error reading message error="websocket: close 1006 (abnormal closure): unexpected EOF"
whishper-backend        | 7:26AM ERR Error reading message error="websocket: close 1006 (abnormal closure): unexpected EOF"
whishper-backend        | 7:27AM ERR Error reading message error="websocket: close 1006 (abnormal closure): unexpected EOF"
whishper-backend        | 7:28AM ERR Error reading message error="websocket: close 1006 (abnormal closure): unexpected EOF"
whishper-backend        | 7:29AM ERR Error reading message error="websocket: close 1006 (abnormal closure): unexpected EOF"
whishper-nginx-1        | 172.18.0.1 - - [02/Nov/2023:07:17:24 +0000] "GET /api/transcriptions HTTP/1.1" 200 1815534 "http://localhost:8082/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36"
whishper-nginx-1        | 172.18.0.1 - - [02/Nov/2023:07:17:24 +0000] "GET /languages HTTP/1.1" 200 175 "http://localhost:8082/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36"
whishper-nginx-1        | 172.18.0.1 - - [02/Nov/2023:07:17:41 +0000] "POST /api/transcriptions HTTP/1.1" 200 0 "http://localhost:8082/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36"
whishper-nginx-1        | 172.18.0.1 - - [02/Nov/2023:07:18:51 +0000] "GET /ws/transcriptions HTTP/1.1" 101 874 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36"
whishper-nginx-1        | 172.18.0.1 - - [02/Nov/2023:07:19:53 +0000] "GET /ws/transcriptions HTTP/1.1" 101 0 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36"
whishper-nginx-1        | 172.18.0.1 - - [02/Nov/2023:07:20:55 +0000] "GET /ws/transcriptions HTTP/1.1" 101 0 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36"
whishper-nginx-1        | 172.18.0.1 - - [02/Nov/2023:07:21:57 +0000] "GET /ws/transcriptions HTTP/1.1" 101 0 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36"
whishper-nginx-1        | 172.18.0.1 - - [02/Nov/2023:07:22:59 +0000] "GET /ws/transcriptions HTTP/1.1" 101 0 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36"
whishper-nginx-1        | 172.18.0.1 - - [02/Nov/2023:07:24:01 +0000] "GET /ws/transcriptions HTTP/1.1" 101 0 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36"
whishper-nginx-1        | 172.18.0.1 - - [02/Nov/2023:07:25:03 +0000] "GET /ws/transcriptions HTTP/1.1" 101 0 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36"
whishper-nginx-1        | 172.18.0.1 - - [02/Nov/2023:07:26:05 +0000] "GET /ws/transcriptions HTTP/1.1" 101 0 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36"
whishper-nginx-1        | 172.18.0.1 - - [02/Nov/2023:07:27:06 +0000] "GET /ws/transcriptions HTTP/1.1" 101 0 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36"
whishper-nginx-1        | 172.18.0.1 - - [02/Nov/2023:07:28:08 +0000] "GET /ws/transcriptions HTTP/1.1" 101 0 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36"
whishper-nginx-1        | 172.18.0.1 - - [02/Nov/2023:07:29:10 +0000] "GET /ws/transcriptions HTTP/1.1" 101 0 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36"
whishper-nginx-1        | 172.18.0.1 - - [02/Nov/2023:07:30:12 +0000] "GET /ws/transcriptions HTTP/1.1" 101 0 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36"
whishper-nginx-1        | 172.18.0.1 - - [02/Nov/2023:07:31:14 +0000] "GET /ws/transcriptions HTTP/1.1" 101 0 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36"
whishper-nginx-1        | 172.18.0.1 - - [02/Nov/2023:07:32:16 +0000] "GET /ws/transcriptions HTTP/1.1" 101 0 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36"
whisper-libretranslate  | Updating language models
whisper-libretranslate  | Found 86 models
whisper-libretranslate  | Keep 4 models
whisper-libretranslate  | Downloading English → French (1.0) ...
whisper-libretranslate  | Downloading English → Spanish (1.0) ...
whisper-libretranslate  | Downloading French → English (1.0) ...
whisper-libretranslate  | Downloading Spanish → English (1.0) ...
whisper-libretranslate  | Loaded support for 3 languages (4 models total)!
whisper-libretranslate  | Running on http://0.0.0.0:5000
whishper-backend        | 7:30AM ERR Error reading message error="websocket: close 1006 (abnormal closure): unexpected EOF"
whisper-api             |   File "/usr/local/lib/python3.10/dist-packages/starlette/routing.py", line 718, in __call__
whishper-web            | Listening on 0.0.0.0:3000
whishper-backend        | 7:31AM ERR Error reading message error="websocket: close 1006 (abnormal closure): unexpected EOF"
whishper-backend        | 7:32AM ERR Error reading message error="websocket: close 1006 (abnormal closure): unexpected EOF"
whisper-api             |     await route.handle(scope, receive, send)
whisper-api             |   File "/usr/local/lib/python3.10/dist-packages/starlette/routing.py", line 276, in handle
whisper-api             |     await self.app(scope, receive, send)
whisper-api             |   File "/usr/local/lib/python3.10/dist-packages/starlette/routing.py", line 66, in app
whisper-api             |     response = await func(request)
whisper-api             |   File "/usr/local/lib/python3.10/dist-packages/fastapi/routing.py", line 273, in app
whisper-api             |     raw_response = await run_endpoint_function(
whisper-api             |   File "/usr/local/lib/python3.10/dist-packages/fastapi/routing.py", line 190, in run_endpoint_function
whisper-api             |     return await dependant.call(**values)
whisper-api             |   File "/app/main.py", line 26, in transcribe_endpoint
whisper-api             |     return await transcribe_file(file, model_size.value, language.value, device)
whisper-api             |   File "/app/transcribe.py", line 41, in transcribe_file
whisper-api             |     return await transcribe_audio(audio, model_size, language, device)
whisper-api             |   File "/app/transcribe.py", line 54, in transcribe_audio
whisper-api             |     model.load()
whisper-api             |   File "/app/backends/fasterwhisper.py", line 29, in load
whisper-api             |     self.model = WhisperModel(
whisper-api             |   File "/usr/local/lib/python3.10/dist-packages/faster_whisper/transcribe.py", line 128, in __init__
whisper-api             |     self.model = ctranslate2.models.Whisper(
whisper-api             | ValueError: Requested int8 compute type, but the target device or backend do not support efficient int8 computation.
whisper-api             | INFO:     127.0.0.1:51714 - "GET /healthcheck/ HTTP/1.1" 200 OK
whisper-api             | INFO:     127.0.0.1:39374 - "GET /healthcheck/ HTTP/1.1" 200 OK
whisper-api             | INFO:     127.0.0.1:45108 - "GET /healthcheck/ HTTP/1.1" 200 OK
whisper-api             | INFO:     127.0.0.1:46154 - "GET /healthcheck/ HTTP/1.1" 200 OK
whisper-api             | INFO:     127.0.0.1:42326 - "GET /healthcheck/ HTTP/1.1" 200 OK
whisper-api             | INFO:     127.0.0.1:53210 - "GET /healthcheck/ HTTP/1.1" 200 OK
whisper-api             | INFO:     127.0.0.1:45252 - "GET /healthcheck/ HTTP/1.1" 200 OK
whisper-api             | INFO:     127.0.0.1:56202 - "GET /healthcheck/ HTTP/1.1" 200 OK
whisper-api             | INFO:     127.0.0.1:59880 - "GET /healthcheck/ HTTP/1.1" 200 OK
whisper-api             | INFO:     127.0.0.1:55626 - "GET /healthcheck/ HTTP/1.1" 200 OK
whisper-api             | INFO:     127.0.0.1:47922 - "GET /healthcheck/ HTTP/1.1" 200 OK
whisper-api             | INFO:     127.0.0.1:48724 - "GET /healthcheck/ HTTP/1.1" 200 OK
whisper-api             | INFO:     127.0.0.1:50112 - "GET /healthcheck/ HTTP/1.1" 200 OK
whisper-api             | INFO:     127.0.0.1:44196 - "GET /healthcheck/ HTTP/1.1" 200 OK
whisper-api             | INFO:     127.0.0.1:40242 - "GET /healthcheck/ HTTP/1.1" 200 OK
whisper-api             | INFO:     127.0.0.1:36154 - "GET /healthcheck/ HTTP/1.1" 200 OK
whisper-api             | INFO:     127.0.0.1:43744 - "GET /healthcheck/ HTTP/1.1" 200 OK
whisper-api             | INFO:     127.0.0.1:33182 - "GET /healthcheck/ HTTP/1.1" 200 OK
whisper-api             | INFO:     127.0.0.1:45358 - "GET /healthcheck/ HTTP/1.1" 200 OK
whisper-api             | INFO:     127.0.0.1:41764 - "GET /healthcheck/ HTTP/1.1" 200 OK
whisper-api             | INFO:     127.0.0.1:37248 - "GET /healthcheck/ HTTP/1.1" 200 OK
whisper-api             | INFO:     127.0.0.1:54336 - "GET /healthcheck/ HTTP/1.1" 200 OK
whisper-api             | INFO:     127.0.0.1:48916 - "GET /healthcheck/ HTTP/1.1" 200 OK
whisper-api             | INFO:     127.0.0.1:49806 - "GET /healthcheck/ HTTP/1.1" 200 OK
whisper-api             | INFO:     127.0.0.1:44938 - "GET /healthcheck/ HTTP/1.1" 200 OK
whisper-api             | INFO:     127.0.0.1:49610 - "GET /healthcheck/ HTTP/1.1" 200 OK
whisper-api             | INFO:     127.0.0.1:40232 - "GET /healthcheck/ HTTP/1.1" 200 OK
whisper-api             | INFO:     127.0.0.1:57346 - "GET /healthcheck/ HTTP/1.1" 200 OK
whisper-api             | INFO:     127.0.0.1:45674 - "GET /healthcheck/ HTTP/1.1" 200 OK

Docker Compose File

version: "3.9"

services:
  mongo:
    image: mongo
    env_file:
      - .env
    restart: unless-stopped
    volumes:
      - ./whishper_data/db_data:/data/db
      - ./whishper_data/db_data/logs/:/var/log/mongodb/
    environment:
      MONGO_INITDB_ROOT_USERNAME: ${DB_USER:-whishper}
      MONGO_INITDB_ROOT_PASSWORD: ${DB_PASS:-whishper}
    expose:
      - 27017
    command: ['--logpath', '/var/log/mongodb/mongod.log']
    healthcheck:
      test: mongosh --eval 'db.runCommand("ping").ok' --quiet
      interval: 2s
      timeout: 3s
      retries: 5

  translate-cpu:
    container_name: whisper-libretranslate
    profiles: ["cpu"]
    image: libretranslate/libretranslate:latest
    restart: unless-stopped
    volumes:
      - ./whishper_data/libretranslate/data:/home/libretranslate/.local/share
      - ./whishper_data/libretranslate/cache:/home/libretranslate/.local/cache
    env_file:
      - .env
    tty: true
    environment:
      LT_DISABLE_WEB_UI: True
      #LT_UPDATE_MODELS: True
    expose:
      - 5000
    networks:
      default:
        aliases:
          - translate
    healthcheck:
      test: ['CMD-SHELL', './venv/bin/python scripts/healthcheck.py']
      interval: 2s
      timeout: 3s
      retries: 5

  translate-gpu:
    container_name: whisper-libretranslate
    image: libretranslate/libretranslate:latest-cuda
    profiles: ["gpu"]
    restart: unless-stopped
    volumes:
      - ./whishper_data/libretranslate/data:/home/libretranslate/.local/share
      - ./whishper_data/libretranslate/cache:/home/libretranslate/.local/cache
    env_file:
      - .env
    user: root
    tty: true
    environment:
      LT_DISABLE_WEB_UI: True
      LT_LOAD_ONLY: ${LT_LOAD_ONLY:-en,fr,es}
      #LT_UPDATE_MODELS: True
    expose:
      - 5000
    networks:
      default:
        aliases:
          - translate
    deploy:
      resources:
        reservations:
          devices:
          - driver: nvidia
            count: all
            capabilities: [gpu]
    healthcheck:
      test: ['CMD-SHELL', './venv/bin/python scripts/healthcheck.py']
      interval: 2s
      timeout: 3s
      retries: 5

  transcription-api-cpu:
    container_name: whisper-api
    profiles: ["cpu"]
    image: pluja/whishper-transcription-api:latest
    env_file:
      - .env
    volumes:
      - ./whishper_data/whisper_models:/app/models
      - ./whishper_data/uploads:/app/uploads
    environment:
      - WHISPER_MODELS_DIR=/app/models
      - UPLOAD_DIR=/app/uploads
    networks:
      default:
        aliases:
          - transcription-api
    restart: unless-stopped
    expose:
      - 8000

  transcription-api-gpu:
    container_name: whisper-api
    profiles: ["gpu"]
    image: pluja/whishper-transcription-api:latest-gpu
    env_file:
      - .env
    volumes:
      - ./whishper_data/whisper_models:/app/models
      - ./whishper_data/uploads:/app/uploads
    environment:
      WHISPER_MODELS_DIR: /app/models
      UPLOAD_DIR: /app/uploads
    restart: unless-stopped
    expose:
      - 8000
    networks:
      default:
        aliases:
          - transcription-api
    deploy:
      resources:
        reservations:
          devices:
          - driver: nvidia
            count: all
            capabilities: [gpu]

  whishper-backend:
    container_name: whishper-backend
    image: pluja/whishper-backend:latest
    env_file:
      - .env
    volumes:
      - ./whishper_data/uploads:/app/uploads
    restart: unless-stopped
    depends_on:
      mongo:
        condition: service_healthy
    healthcheck:
      test: curl --fail http://localhost:8080/api/transcriptions || exit 1
      interval: 2s
      timeout: 2s
      retries: 5
    expose:
      - 8080

  whishper-frontend:
    container_name: whishper-web
    image: pluja/whishper-frontend:latest
    env_file:
      - .env
    restart: unless-stopped
    environment:
      PUBLIC_INTERNAL_API_HOST: "http://nginx:80"
      PUBLIC_TRANSLATION_API_HOST: ""
      PUBLIC_API_HOST: ${WHISHPER_HOST:-}
      PUBLIC_WHISHPER_PROFILE: ${COMPOSE_PROFILES:-false}
    expose:
      - 3000

  nginx:
    image: nginx:1
    restart: unless-stopped
    volumes:
      - ./nginx.conf:/etc/nginx/nginx.conf
      - ./whishper_data/uploads:/app/uploads
    depends_on:
      whishper-backend:
        condition: service_healthy
    ports:
      - 8082:80

maccaveli avatar Nov 02 '23 07:11 maccaveli