Kokoro-FastAPI Why use cuda12.8 ? why not support more vesion?

Mar 03 '25 08:03 soul667

Yes, it would be great to have CUDA 12.7 support. Nvidia Windows drivers what came with CUDA 12.8 and RTX 5000 support are buggy. Last stable driver release is from December.

Mar 05 '25 20:03 AlichinoML

Same problem here. Please lower the CUDA version. The Nvidia drivers for windows that have CUDA 12.8 are buggy af...

Mar 08 '25 19:03 reeson46

LTS drivers from nvidia on linux are 535.*, they support CUDA 12.2. This is the cuda version that NVIDIA uses in their own docker images

Mar 18 '25 22:03 synchrone

Yup. I'm unable to use this because our system has NVIDIA driver version 565.77 but is on CUDA 12.7. After building and getting the Docker container ready for Kokoro I was stopped in my tracks because it wanted CUDA 12.8.

Real bummer ......

Mar 21 '25 12:03 jlar0che

Same here, using Ubuntu server 24.04 the latest drivers are 550.120, which is cuda version 12.4, so unable to continue further.

Mar 25 '25 19:03 nikito

Same here, my system only support up to coda 12.4

Mar 29 '25 17:03 MwC-Trexx

Yeah, please push the version support down lower as other dockers I run require using an older version such as 12.6

Apr 06 '25 04:04 Vahnil

CUDA 12.4 here, working perfectly, direct run

Apr 10 '25 11:04 SunixLiu

CUDA 12.4 here, working perfectly, direct run

Yeah, think the issue is that the docker container requires 12.8, so people trying to run this in docker or k8s run into problems.

Apr 10 '25 12:04 nikito

still an issue on my end, i have to run the CPU version, because of cuda issues, ollama and other dockers have no issues seeing and using the GPU

Apr 12 '25 13:04 Array-zer0

I fixed the issue on my end. I changed the first line of the dockerfile from

FROM --platform=$BUILDPLATFORM nvidia/cuda:12.8.0-cudnn-runtime-ubuntu24.04

to

FROM --platform=$BUILDPLATFORM nvidia/cuda:12.6.0-cudnn-runtime-ubuntu24.04

And it worked for my situation, it will likely work for lower cuda versions as well.

Apr 13 '25 01:04 Vahnil

I agree that this needs to be changed as I am also having issues with it. I did get the container to run by adding these two lines to my environment in compose:

NVIDIA_REQUIRE_CUDA=cuda>=12.4
CUDA_VERSION=12.4.0

That said, I downgraded to v0.2.2 as well.

Apr 23 '25 20:04 GeekMage

I agree that this needs to be changed as I am also having issues with it. I did get the container to run by adding these two lines to my environment in compose:

NVIDIA_REQUIRE_CUDA=cuda>=12.4

CUDA_VERSION=12.4.0

That said, I downgraded to v0.2.2 as well.

Can this be done to make cuda12.4 run properly? I am currently using CUDA 12.4 version. I also hope to find a solution to utilize kokoro.

Apr 27 '25 06:04 WThirteen

I agree that this needs to be changed as I am also having issues with it. I did get the container to run by adding these two lines to my environment in compose:

NVIDIA_REQUIRE_CUDA=cuda>=12.4

CUDA_VERSION=12.4.0

That said, I downgraded to v0.2.2 as well.

Can this be done to make cuda12.4 run properly? I am currently using CUDA 12.4 version. I also hope to find a solution to utilize kokoro.

I am running it now using those environment variables on TrueNAS Scale and docker.

Apr 27 '25 11:04 GeekMage

I agree that this needs to be changed as I am also having issues with it. I did get the container to run by adding these two lines to my environment in compose:

NVIDIA_REQUIRE_CUDA=cuda>=12.4

CUDA_VERSION=12.4.0

That said, I downgraded to v0.2.2 as well.

Can this be done to make cuda12.4 run properly? I am currently using CUDA 12.4 version. I also hope to find a solution to utilize kokoro.

I am running it now using those environment variables on TrueNAS Scale and docker.

Thank you. I'm currently attempting to package the Docker image. Use CUDA 12.4

Apr 28 '25 01:04 WThirteen

I agree that this needs to be changed as I am also having issues with it. I did get the container to run by adding these two lines to my environment in compose:

NVIDIA_REQUIRE_CUDA=cuda>=12.4

CUDA_VERSION=12.4.0

That said, I downgraded to v0.2.2 as well.

Can this be done to make cuda12.4 run properly? I am currently using CUDA 12.4 version. I also hope to find a solution to utilize kokoro.

I am running it now using those environment variables on TrueNAS Scale and docker.

I have the same setup, but it seems it doesn't use GPU.

Sep 04 '25 05:09 yosoufe

I agree that this needs to be changed as I am also having issues with it. I did get the container to run by adding these two lines to my environment in compose:

NVIDIA_REQUIRE_CUDA=cuda>=12.4

CUDA_VERSION=12.4.0

That said, I downgraded to v0.2.2 as well.

Can this be done to make cuda12.4 run properly? I am currently using CUDA 12.4 version. I also hope to find a solution to utilize kokoro.

I am running it now using those environment variables on TrueNAS Scale and docker.

I have the same setup, but it seems it doesn't use GPU.

The response rate of this TTS is extremely fast. Even with the response time of the CPU, it is still acceptable.

Sep 04 '25 07:09 WThirteen

I agree that this needs to be changed as I am also having issues with it. I did get the container to run by adding these two lines to my environment in compose:

NVIDIA_REQUIRE_CUDA=cuda>=12.4

CUDA_VERSION=12.4.0

That said, I downgraded to v0.2.2 as well.

Can this be done to make cuda12.4 run properly? I am currently using CUDA 12.4 version. I also hope to find a solution to utilize kokoro.

I am running it now using those environment variables on TrueNAS Scale and docker.

I have the same setup, but it seems it doesn't use GPU.

The response rate of this TTS is extremely fast. Even with the response time of the CPU, it is still acceptable.

In case you wanted the GPU support for even faster TTS for your TrueNas I created an image that can be pulled from ghcr.io/yosoufe/kokoro-fastapi:1.0. It is using cuda 12.4. It is noticeably faster than CPU.

compose file would look like

version: '3.8'

services:
  kokoro-fastapi-gpu:
    ports:
      - 8880:8880
    image: ghcr.io/yosoufe/kokoro-fastapi:1.0
    restart: always
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities:
                - gpu

Sep 04 '25 17:09 yosoufe

I agree that this needs to be changed as I am also having issues with it. I did get the container to run by adding these two lines to my environment in compose:

NVIDIA_REQUIRE_CUDA=cuda>=12.4

CUDA_VERSION=12.4.0

That said, I downgraded to v0.2.2 as well.

Can this be done to make cuda12.4 run properly? I am currently using CUDA 12.4 version. I also hope to find a solution to utilize kokoro.

I am running it now using those environment variables on TrueNAS Scale and docker.

I have the same setup, but it seems it doesn't use GPU.

The response rate of this TTS is extremely fast. Even with the response time of the CPU, it is still acceptable.

In case you wanted the GPU support for even faster TTS for your TrueNas I created an image that can be pulled from ghcr.io/yosoufe/kokoro-fastapi:1.0. It is using cuda 12.4. It is noticeably faster than CPU.

compose file would look like

version: '3.8'

services: kokoro-fastapi-gpu: ports: - 8880:8880 image: ghcr.io/yosoufe/kokoro-fastapi:1.0 restart: always deploy: resources: reservations: devices: - driver: nvidia count: all capabilities: - gpu

That’s great!

I wanted to mention that after my original comment, I forked the repo and changed a single value in the config and then built the image myself. It was good for the most current version.

If anyone wants to fork their own or just use mine, you can look and see what I did.

Sep 04 '25 21:09 GeekMage