THUDM
[BUG/Help] <我不得不说> 在配置不错的情况下，运行也慢，官方 demo respone 也是空的
Is there an existing issue for this?
- [X] I have searched the existing issues
Current Behavior
配置：
RTX 3090，显存 24G
12核 CPU
内存：43G
运行 api.py。回答很慢，最后回答出的内容，还是空的。respone 空
Expected Behavior
在配置可以的情况下，速度快一些。内容能出来
Steps To Reproduce
- clone 项目并配置好一切；
- 运行 api.py
- 去进行模拟请求
Environment
- OS:Ubuntu 20.04
- Python: 3.8
- Transformers: 4.27.1
- PyTorch: 1.12
- CUDA Support (`python -c "import torch; print(torch.cuda.is_available())"`) : True
Anything else?
No response
@duzx16 如下。
root@autodl-container-bf3b118d52-9a3a8d5f:~/autodl-tmp# nvidia-smi Fri Apr 28 16:32:08 2023 +-----------------------------------------------------------------------------+ | NVIDIA-SMI 515.65.01 Driver Version: 515.65.01 CUDA Version: 11.7 | |-------------------------------+----------------------+----------------------+ | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |===============================+======================+======================| | 0 NVIDIA GeForce ... On | 00000000:8A:00.0 Off | N/A | | 0% 26C P8 18W / 350W | 12821MiB / 24576MiB | 0% Default | | | | N/A | +-------------------------------+----------------------+----------------------+
@duzx16 如下。
root@autodl-container-bf3b118d52-9a3a8d5f:~/autodl-tmp# nvidia-smi Fri Apr 28 16:32:08 2023 +-----------------------------------------------------------------------------+ | NVIDIA-SMI 515.65.01 Driver Version: 515.65.01 CUDA Version: 11.7 | |-------------------------------+----------------------+----------------------+ | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |===============================+======================+======================| | 0 NVIDIA GeForce ... On | 00000000:8A:00.0 Off | N/A | | 0% 26C P8 18W / 350W | 12821MiB / 24576MiB | 0% Default | | | | N/A | +-------------------------------+----------------------+----------------------+
@duzx16
我这里 windows 11下 80GB内存， GTX1080ti 11GB，Driver Version: 531.61, CUDA Version: 12.1，跑官方的web_demo.py, 量化8bit，感觉速度还可以。
我这里 windows 11下 80GB内存， GTX1080ti 11GB，Driver Version: 531.61, CUDA Version: 12.1，跑官方的web_demo.py, 量化8bit，感觉速度还可以。
good 的
