JittorLLMs

JittorLLMs copied to clipboard

ithaos

CUDA found but cublas is not loaded

1

在跑`python cli_demo.py llama`的时候出问题 ``` (stable_vicuna) PS G:\stable_vicuna\JittorLLMs> python cli_demo.py llama [i 0603 18:44:57.225000 56 compiler.py:955] Jittor(1.3.7.16) src: d:\anaconda\lib\site-packages\jittor [i 0603 18:44:57.242000 56 compiler.py:956] cl at G:\stable_vicuna\jittor_home\.cache\jittor\msvc\VC\_\_\_\_\_\bin\cl.exe(19.29.30133) [i 0603 18:44:57.242000 56...

Hihoshi

盘古模型几乎是玩具模型。chatglm好用不少

jellchou

chatglm可不可以8bit量化

3

我是3060M（6G）+16G内存的，跑chatglm能把内存显存统统吃干抹净，有没有办法进行8bit量化？

YIZXIY

用现有代码跑 web_demo.py 时速度很慢，经定位发现是因为 models/chatglm/\_\_init\_\_.py 中 run_web_demo 调用 run 使用了 chat 接口 为换成调用 stream_chat 接口，在 web_demo.py 里强行将代码修改如下是可以正常使用的： ````py # for response, history in model.run_web_demo(input, history): for response, history in model.model.stream_chat(model.tokenizer, input,...

AIxyz

No module named 'torch.utils.hooks'

5

ModuleNotFoundError: No module named 'torch.utils.hooks'

chalecao

The error happens when loading checkpoint shards: 50%|████████ | 4/8 python3.10/site-packages/torch/serialization.py: ││ 283 in __init__ ││ ││ 280 ││ 281 class _open_zipfile_reader(_opener): ││ 282 │ def __init__(self, name_or_buffer) -> None:...

billzhao9

Mac M1 pro运行后错误如下 请大神指导，谢谢

1

(jittor) lwo2002@AlexludeMacBook-Pro JittorLLMs % python web_demo.py chatglm Traceback (most recent call last): File "/Users/lwo2002/JittorLLMs/web_demo.py", line 26, in model = models.get_model(args) File "/Users/lwo2002/JittorLLMs/models/__init__.py", line 38, in get_model globals()[f"get_{model_name}"]() File "/Users/lwo2002/JittorLLMs/models/util.py", line...

lwo2002

缓存数据不能复用，镜像方式每次构建会重新下载，造成资源浪费与耗时，是否有比较好的方法处理这个问题呢？谢谢  占用空间从最开始的20G不到，构建了几次，耗时太久，然后越来越大 

xwi88

LawrenceHan