remsky
Blackwell not supported yet?
Launching the docker:
WARNING: Defaulting repo_id to hexgrad/Kokoro-82M. Pass repo_id='hexgrad/Kokoro-82M' to suppress this warning.
/app/.venv/lib/python3.10/site-packages/torch/nn/modules/rnn.py:123: UserWarning: dropout option adds dropout after all but last recurrent layer, so non-zero dropout expects num_layers greater than 1, but got dropout=0.2 and num_layers=1
warnings.warn(
/app/.venv/lib/python3.10/site-packages/torch/nn/utils/weight_norm.py:143: FutureWarning: torch.nn.utils.weight_norm is deprecated in favor of torch.nn.utils.parametrizations.weight_norm.
WeightNorm.apply(module, name, dim)
/app/.venv/lib/python3.10/site-packages/torch/cuda/init.py:235: UserWarning:
NVIDIA RTX PRO 6000 Blackwell Workstation Edition with CUDA capability sm_120 is not compatible with the current PyTorch installation.
The current PyTorch install supports CUDA capabilities sm_50 sm_60 sm_70 sm_75 sm_80 sm_86 sm_90.
If you want to use the NVIDIA RTX PRO 6000 Blackwell Workstation Edition GPU with PyTorch, please check the instructions at https://pytorch.org/get-started/locally/
I've seen this error plenty of times and it's usually some package, torch or otherwise, that isn't updated for Blackwell compatibility (RTX 5060-5090, 6000).
FYI I was able to get kokoro to launch and work by updating torch inside the container:
python -m ensurepip --default-pip python -m pip uninstall torch python -m pip install --upgrade torch --index-url https://download.pytorch.org/whl/cu128
However, this messed up something with being able to actually launch the container. The container just immediately stops, no log, no nothing. I have to launch the container in interactive mode and manually run the entry script command:
In host: docker run -it --name kokoro --entrypoint=/bin/bash -p 8880:8880 --gpus all kokoro-fastapi-gpu-updated
In the launched container interactive prompt: ./entrypoint.sh
which does work.. but it's kind of annoying to not just be able to click a button in the docker UI to start the container.
Anyway, it proved that updating torch does allow kokoro in the docker to load on my Blackwell card, not sure why the container doesn't launch normally anymore, though.
Fixed the issue with restarting it. I wasn't creating the new container with replacing the bash entrypoint I had used to be able to update the container, so the new container was just relaunching the bash instead. Launching the updated container with --entrypoint=/app/entrypoint.sh fixed it.
So yeah, if you want to update the official docker to fully support Blackwell all you should need to do on the existing container is have the latest torch per my command above.
Until a new release is cut you can use ghcr.io/rushyrush/kokoro-fastapi-gpu:v0.3.0.
I forked from #390 to get 5000 series cards working.
