按照#65中的解答 源码安装后Triton 2.1.0 后报错

[Chun-QiuCC]

起初报错内容和#65报错内容一样，源码安装后再次运行报错如下：

(moss) root@autodl-container-99da11befa-9a05a134:~/autodl-tmp/MOSS/MOSS# python moss_gui_demo.py
Waiting for all devices to be ready, it may take a few minutes...
triton not installed. Run `pip install triton` to load quantized version of MOSS.
Traceback (most recent call last):
  File "moss_gui_demo.py", line 31, in <module>
    raw_model = MossForCausalLM._from_config(config, torch_dtype=torch.float16)
  File "/root/miniconda3/envs/moss/lib/python3.8/site-packages/transformers/modeling_utils.py", line 1024, in _from_config
    model = cls(config, **kwargs)
  File "/root/autodl-tmp/MOSS/MOSS/models/modeling_moss.py", line 612, in __init__
    self.quantize(config.wbits, config.groupsize)
  File "/root/autodl-tmp/MOSS/MOSS/models/modeling_moss.py", line 736, in quantize
    from .quantization import quantize_with_gptq
  File "/root/autodl-tmp/MOSS/MOSS/models/quantization.py", line 27, in <module>
    @autotune(
NameError: name 'autotune' is not defined

已删除autotune后重新安装，依旧报错

[Chun-QiuCC]

按照#60的方法修改 try: import triton import triton.language as tl from .custom_autotune import * except: 改成 try: import triton import triton.language as tl from custom_autotune import * except: 还是会报错

[Chen477777]

还需要将custom_autotune.py文件放入缓存中，参考位置是~/.cache/huggingface/modules/transformers_modules/local/

[Chun-QiuCC]

还需要将custom_autotune.py文件放入缓存中，参考位置是~/.cache/huggingface/modules/transformers_modules/local/

我没有找到这个文件夹，我是手动从huggingface下载的模型

[937739823]

还需要将custom_autotune.py文件放入缓存中，参考位置是~/.cache/huggingface/modules/transformers_modules/local/

没有local把，linux下路径是 ~/.cache/huggingface/modules/transformers_modules/fnlp/moss-moon-003-sft-int4/ 哪里有local

[zay95]

遇到了同样的问题，NameError: name 'autotune' is not defined, 把下载的模型的文件custom_autotune.py复制到 /home/XXXXX/.cache/huggingface/modules/transformers_modules/moss-moon-003-sft-int4下面，这个文件夹一般运行过一次代码就会生成； triton从源码安装，可以运行了

[linonetwo]

使用 docker + WSL2 的话，没有 /home/xxx ，home 目录直接是空的

[linonetwo]

根据报错内容

  File "/root/.cache/huggingface/modules/transformers_modules/local/quantization.py", line 27, in <module>
    @autotune(
NameError: name 'autotune' is not defined

可知在 docker 里文件夹应该是 "/root/.cache/huggingface/modules/transformers_modules/local/

执行这个即可

cp /mnt/llm/MOSS/moss-moon-003-sft-plugin-int4/custom_autotune.py /root/.cache/huggingface/modules/transformers_modules/local/