generate: AttributeError: 'NoneType' object has no attribute 'eval'

[Tian14267]

我在运行 generate.sh的时候，提示报错：

Traceback (most recent call last):
  File "generate.py", line 110, in <module>
    model.eval()
AttributeError: 'NoneType' object has no attribute 'eval'

我检查了代码，问题出在这里： 当把这些注释之后，就可以了。但是却无法使用 stream_generate。请问这是啥情况

[Tian14267]

在进行finetune的时候也遇到类似的问题：

Traceback (most recent call last):
  File "finetune.py", line 132, in <module>
    model.print_trainable_parameters()
AttributeError: 'NoneType' object has no attribute 'print_trainable_parameters'
ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: 1) local_rank: 0 (pid: 81689) of binary: /root/anaconda3/envs/Belle/bin/python
Traceback (most recent call last):
  File "/root/anaconda3/envs/Belle/bin/torchrun", line 8, in <module>
    sys.exit(main())
  File "/root/anaconda3/envs/Belle/lib/python3.8/site-packages/torch/distributed/elastic/multiprocessing/errors/__init__.py", line 346, in wrapper
    return f(*args, **kwargs)
  File "/root/anaconda3/envs/Belle/lib/python3.8/site-packages/torch/distributed/run.py", line 762, in main
    run(args)
  File "/root/anaconda3/envs/Belle/lib/python3.8/site-packages/torch/distributed/run.py", line 753, in run
    elastic_launch(
  File "/root/anaconda3/envs/Belle/lib/python3.8/site-packages/torch/distributed/launcher/api.py", line 132, in __call__
    return launch_agent(self._config, self._entrypoint, list(args))
  File "/root/anaconda3/envs/Belle/lib/python3.8/site-packages/torch/distributed/launcher/api.py", line 246, in launch_agent
    raise ChildFailedError(
torch.distributed.elastic.multiprocessing.errors.ChildFailedError: 
============================================================
finetune.py FAILED
------------------------------------------------------------
Failures:
  <NO_OTHER_FAILURES>
------------------------------------------------------------
Root Cause (first observed failure):
[0]:
  time      : 2023-04-13_15:37:58
  host      : gpu19
  rank      : 0 (local_rank: 0)
  exitcode  : 1 (pid: 81689)
  error_file: <N/A>
  traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html
============================================================

就好像是model是None

[xqmmy]

peft版本的问题

[yanxp]

@xqmmy @Tian14267 这个问题解决了嘛 遇到类似的问题，peft用哪个版本呢，用的是推荐版本

[xqmmy]

@xqmmy @Tian14267 这个问题解决了嘛 遇到类似的问题，peft用哪个版本呢，用的是推荐版本

#51 看下这个

[Facico]

@yanxp 我们在最新的requirement.txt中的固定的某个commit hash的版本应该是没有问题的git+https://github.com/huggingface/peft@e536616888d51b453ed354a6f1e243fecb02ea08

[mrbemani]

用了最新的版本PEFT还是同样的错误。

(textgen) [shiqi@nqs-server Chinese-Vicuna]$ ./generate.sh 
bin /home/shiqi/anaconda3/envs/textgen/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cuda117.so
Namespace(model_path='decapoda-research/llama-7b-hf', lora_path='Chinese-Vicuna/Chinese-Vicuna-lora-7b-belle-and-guanaco', use_typewriter=0, use_local=0)
The tokenizer class you load from this checkpoint is not the same type as the class this function is called from. It may result in unexpected tokenization. 
The tokenizer class you load from this checkpoint is 'LLaMATokenizer'. 
The class this function is called from is 'LlamaTokenizer'.
Chinese-Vicuna/Chinese-Vicuna-lora-7b-belle-and-guanaco/adapter_model.bin
Loading checkpoint shards: 100%|████████████████████████████████████████████████████████████████████████| 33/33 [00:12<00:00,  2.68it/s]Traceback (most recent call last):
  File "/mnt/d/tools/nlp/Chinese-Vicuna/generate.py", line 110, in <module>
    model.eval()
AttributeError: 'NoneType' object has no attribute 'eval'

好像是Pytorch 2.0的，问题，换成1.13就正常了。

[Facico]

@mrbemani 但是不应该是最新版吧，这个问题就是因为PEFT一直在更新导致的问题。所以我们现在是固定了以前的一个版本。

[mrbemani]

@mrbemani 但是不应该是最新版吧，这个问题就是因为PEFT一直在更新导致的问题。所以我们现在是固定了以前的一个版本。

是最新版的Chinese-Vicuna，标点符号忘记打了，哈哈

[Facico]

@mrbemani 那你可以把peft版本卸了重装一下

!pip uninstall peft
!pip install git+https://github.com/huggingface/peft@e536616888d51b453ed354a6f1e243fecb02ea08

[Tian14267]

@Facico @yanxp 换成 peft==0.2.0 就行了

[niuhuluzhihao]

@Facico @yanxp 换成 peft==0.2.0 就行了 这个是有用的