Does not work on paperspace Quadro 5000 RTX x 2

[ashhadahsan]

I have a setup of Quadro RTX 5000 x2 (16x2) GB GPU RAM, everything is up but no response is getting produced.

Build container using:

sudo CUDA_ARCH=89-real docker compose build

The logs are:

nginx-1          | 51.36.220.128 - - [05/Mar/2024:16:32:24 +0000] "GET /css/all.min.css HTTP/1.1" 304 0 "http://184.105.215.27:8000/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36 Edg/122.0.0.0" "-"
nginx-1          | 51.36.220.128 - - [05/Mar/2024:16:32:24 +0000] "GET /img/microphone-white.png HTTP/1.1" 304 0 "http://184.105.215.27:8000/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36 Edg/122.0.0.0" "-"
nginx-1          | 51.36.220.128 - - [05/Mar/2024:16:32:24 +0000] "GET /js/main.js HTTP/1.1" 304 0 "http://184.105.215.27:8000/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36 Edg/122.0.0.0" "-"
nginx-1          | 51.36.220.128 - - [05/Mar/2024:16:32:24 +0000] "GET /img/record.png HTTP/1.1" 304 0 "http://184.105.215.27:8000/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36 Edg/122.0.0.0" "-"
whisperfusion-1  | INFO:websockets.server:connection open
whisperfusion-1  | INFO:websockets.server:connection open
whisperfusion-1  | INFO:root:[Whisper INFO:] New client connected
nginx-1          | 51.36.220.128 - - [05/Mar/2024:16:32:30 +0000] "GET / HTTP/1.1" 304 0 "-" "Mozilla/5.0 (Linux; Android 6.0; Nexus 5 Build/MRA58N) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Mobile Safari/537.36 Edg/122.0.0.0" "-"
nginx-1          | 51.36.220.128 - - [05/Mar/2024:16:32:30 +0000] "GET /transcription HTTP/1.1" 101 76 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36 Edg/122.0.0.0" "-"
whisperfusion-1  | ERROR:root:received 1001 (going away); then sent 1001 (going away)
whisperfusion-1  | INFO:root:Cleaning up.
whisperfusion-1  | INFO:root:[Whisper INFO:] Connection Closed.
nginx-1          | 51.36.220.128 - - [05/Mar/2024:16:32:30 +0000] "GET /audio HTTP/1.1" 101 0 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36 Edg/122.0.0.0" "-"
whisperfusion-1  | INFO:root:[Whisper INFO:] Exiting speech to text thread
whisperfusion-1  | INFO:websockets.server:connection open
whisperfusion-1  | INFO:websockets.server:connection open
whisperfusion-1  | INFO:root:[Whisper INFO:] New client connected
nginx-1          | 51.36.220.128 - - [05/Mar/2024:16:32:43 +0000] "GET /img/microphone-hover.png HTTP/1.1" 304 0 "http://184.105.215.27:8000/" "Mozilla/5.0 (Linux; Android 6.0; Nexus 5 Build/MRA58N) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Mobile Safari/537.36 Edg/122.0.0.0" "-"
nginx-1          | 51.36.220.128 - - [05/Mar/2024:16:33:07 +0000] "GET / HTTP/1.1" 304 0 "-" "Mozilla/5.0 (Linux; Android 6.0; Nexus 5 Build/MRA58N) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Mobile Safari/537.36 Edg/122.0.0.0" "-"
nginx-1          | 51.36.220.128 - - [05/Mar/2024:16:33:07 +0000] "GET /transcription HTTP/1.1" 101 76 "-" "Mozilla/5.0 (Linux; Android 6.0; Nexus 5 Build/MRA58N) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Mobile Safari/537.36 Edg/122.0.0.0" "-"
nginx-1          | 51.36.220.128 - - [05/Mar/2024:16:33:07 +0000] "GET /audio HTTP/1.1" 101 0 "-" "Mozilla/5.0 (Linux; Android 6.0; Nexus 5 Build/MRA58N) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Mobile Safari/537.36 Edg/122.0.0.0" "-"
whisperfusion-1  | ERROR:root:received 1001 (going away); then sent 1001 (going away)
whisper

What is wrong?

[makaveli10]

Thanks for your interest in WhisperFusion, I think you built docker compose with incorrect compute capability because IIRC compute capability for RTX Quadro RTX 5000 is 7.5 so, to build docker compose correctly:

rm -rf docker/scratch-space/models
docker system prune
CUDA_ARCH=75-real docker compose build

Let us know, if that helps.

[ashhadahsan]

@makaveli10 well now the build is done but it is still not working, there are a few things to consider

1. TensorRT says that it was built on a different version and the installed version is different, it was saying the same thing when I built using 89-real.
2. I am using HTTP and the browser does not allow a microphone over HTTP, so I am also trying to add HTTPS on a custom domain.