peft icon indicating copy to clipboard operation
peft copied to clipboard

Published 20 hours ago • verified publisher icon huggingface

TypeError: PeftConfig.__init__() got an unexpected keyword argument '_name_or_path' 报错怎么解决

Open caijx168 opened this issue 1 year ago • 6 comments

System Info

peft 版本为0.10.0

Who can help?

@pacman100 @younesbelkada @BenjaminBossan @sayakpaul @stevhliu

Information

  • [ ] The official example scripts
  • [ ] My own modified scripts

Tasks

  • [ ] An officially supported task in the examples folder
  • [ ] My own task or dataset (give details below)

Reproduction

部署langchain-chatchat 报错提示这个要怎么解决(Baichuan2) root@root1-System-Product-Name:/home/Baichuan2/Langchain-Chatchat# python startup.py -a

==============================Langchain-Chatchat Configuration============================== 操作系统：Linux-6.5.0-27-generic-x86_64-with-glibc2.35. python版本：3.10.13 (main, Sep 11 2023, 13:44:35) [GCC 11.2.0] 项目版本：v0.2.10 langchain版本：0.0.354. fastchat版本：0.2.35

当前使用的分词器：ChineseRecursiveTextSplitter 当前启动的LLM模型：['peft-HitoGpt'] @ cuda {'device': 'cuda', 'host': '0.0.0.0', 'infer_turbo': False, 'model_path': '/home/Baichuan2/Langchain-Chatchat/peft/peft-HitoGpt', 'model_path_exists': True, 'port': 20002} 当前Embbedings模型： bge-large-zh-v1.5 @ cuda ==============================Langchain-Chatchat Configuration==============================

2024-04-18 09:22:08,920 - startup.py[line:655] - INFO: 正在启动服务： 2024-04-18 09:22:08,920 - startup.py[line:656] - INFO: 如需查看 llm_api 日志，请前往 /home/Baichuan2/Langchain-Chatchat/logs /root/anaconda3/envs/Baichuan2/lib/python3.10/site-packages/langchain_core/_api/deprecation.py:117: LangChainDeprecationWarning: 模型启动功能将于 Langchain-Chatchat 0.3.x重写,支持更多模式和加速启动，0.2.x中相关功能将废弃 warn_deprecated( 2024-04-18 09:22:11 | ERROR | stderr | INFO: Started server process [44838] 2024-04-18 09:22:11 | ERROR | stderr | INFO: Waiting for application startup. 2024-04-18 09:22:11 | ERROR | stderr | INFO: Application startup complete. 2024-04-18 09:22:11 | ERROR | stderr | INFO: Uvicorn running on http://0.0.0.0:20000 (Press CTRL+C to quit) 2024-04-18 09:22:11 | ERROR | stderr | Process model_worker - peft-HitoGpt: 2024-04-18 09:22:11 | ERROR | stderr | Traceback (most recent call last): 2024-04-18 09:22:11 | ERROR | stderr | File "/root/anaconda3/envs/Baichuan2/lib/python3.10/multiprocessing/process.py", line 314, in _bootstrap 2024-04-18 09:22:11 | ERROR | stderr | self.run() 2024-04-18 09:22:11 | ERROR | stderr | File "/root/anaconda3/envs/Baichuan2/lib/python3.10/multiprocessing/process.py", line 108, in run 2024-04-18 09:22:11 | ERROR | stderr | self._target(*self._args, **self._kwargs) 2024-04-18 09:22:11 | ERROR | stderr | File "/home/Baichuan2/Langchain-Chatchat/startup.py", line 389, in run_model_worker 2024-04-18 09:22:11 | ERROR | stderr | app = create_model_worker_app(log_level=log_level, **kwargs) 2024-04-18 09:22:11 | ERROR | stderr | File "/home/Baichuan2/Langchain-Chatchat/startup.py", line 217, in create_model_worker_app 2024-04-18 09:22:11 | ERROR | stderr | worker = ModelWorker( 2024-04-18 09:22:11 | ERROR | stderr | File "/root/anaconda3/envs/Baichuan2/lib/python3.10/site-packages/fastchat/serve/model_worker.py", line 66, in init 2024-04-18 09:22:11 | ERROR | stderr | super().init( 2024-04-18 09:22:11 | ERROR | stderr | File "/root/anaconda3/envs/Baichuan2/lib/python3.10/site-packages/fastchat/serve/base_model_worker.py", line 47, in init 2024-04-18 09:22:11 | ERROR | stderr | self.conv = self.make_conv_template(conv_template, model_path) 2024-04-18 09:22:11 | ERROR | stderr | File "/root/anaconda3/envs/Baichuan2/lib/python3.10/site-packages/fastchat/serve/base_model_worker.py", line 75, in make_conv_template 2024-04-18 09:22:11 | ERROR | stderr | conv = get_conversation_template(model_path) 2024-04-18 09:22:11 | ERROR | stderr | File "/root/anaconda3/envs/Baichuan2/lib/python3.10/site-packages/fastchat/model/model_adapter.py", line 376, in get_conversation_template 2024-04-18 09:22:11 | ERROR | stderr | return adapter.get_default_conv_template(model_path) 2024-04-18 09:22:11 | ERROR | stderr | File "/root/anaconda3/envs/Baichuan2/lib/python3.10/site-packages/fastchat/model/model_adapter.py", line 654, in get_default_conv_template 2024-04-18 09:22:11 | ERROR | stderr | config = PeftConfig.from_pretrained(model_path) 2024-04-18 09:22:11 | ERROR | stderr | File "/root/anaconda3/envs/Baichuan2/lib/python3.10/site-packages/peft/config.py", line 151, in from_pretrained 2024-04-18 09:22:11 | ERROR | stderr | return cls.from_peft_type(**kwargs) 2024-04-18 09:22:11 | ERROR | stderr | File "/root/anaconda3/envs/Baichuan2/lib/python3.10/site-packages/peft/config.py", line 118, in from_peft_type 2024-04-18 09:22:11 | ERROR | stderr | return config_cls(**kwargs) 2024-04-18 09:22:11 | ERROR | stderr | TypeError: PeftConfig.init() got an unexpected keyword argument '_name_or_path'

Expected behavior

帮助我解决问题

caijx168 avatar Apr 18 '24 03:04 caijx168

Sorry, I don't understand everything that you wrote.

There should be an adapter_config.json for your PEFT adapter, probably inside of /home/Baichuan2/Langchain-Chatchat/peft/peft-HitoGpt. Could you paste its content here?

BenjaminBossan avatar Apr 18 '24 09:04 BenjaminBossan

adapter_config.json 如下 cat adapter_config.json { "_name_or_path": "/home/ChatGLM/ChatGLM3/chatglm3-6b", "add_bias_linear": false, "add_qkv_bias": true, "apply_query_key_layer_scaling": true, "apply_residual_connection_post_layernorm": false, "architectures": [ "ChatGLMForConditionalGeneration" ], "attention_dropout": 0.0, "attention_softmax_in_fp32": true, "auto_map": { "AutoConfig": "configuration_chatglm.ChatGLMConfig", "AutoModel": "modeling_chatglm.ChatGLMForConditionalGeneration", "AutoModelForCausalLM": "modeling_chatglm.ChatGLMForConditionalGeneration", "AutoModelForSeq2SeqLM": "modeling_chatglm.ChatGLMForConditionalGeneration", "AutoModelForSequenceClassification": "modeling_chatglm.ChatGLMForSequenceClassification" }, "bias_dropout_fusion": true, "classifier_dropout": null, "eos_token_id": 2, "ffn_hidden_size": 13696, "fp32_residual_connection": false, "hidden_dropout": 0.0, "hidden_size": 4096, "kv_channels": 128, "layernorm_epsilon": 1e-05, "model_type": "chatglm", "multi_query_attention": true, "multi_query_group_num": 2, "num_attention_heads": 32, "num_layers": 28, "original_rope": true, "pad_token_id": 0, "padded_vocab_size": 65024, "post_layer_norm": true, "pre_seq_len": null, "prefix_projection": false, "quantization_bit": 0, "rmsnorm": true, "seq_length": 8192, "tie_word_embeddings": false, "torch_dtype": "float16", "transformers_version": "4.38.2", "use_cache": true, "vocab_size": 65024 }

caijx168 avatar Apr 18 '24 09:04 caijx168

Thanks. This does not look like a adapter_config.json from PEFT. Here is an example of a correct PEFT config file. The file you show looks like a config.json from transformers. Something must have gone wrong with the files when saving the model.

BenjaminBossan avatar Apr 18 '24 10:04 BenjaminBossan

我导出的模型文件是通过LLaMA-Factory微调之后合并导出的，目录结构如下 image 其原本生成的是config.json，然后我将其更名为adapter_config.json

caijx168 avatar Apr 19 '24 06:04 caijx168

Why did you rename config.json to adapter_config.json? Those are not the same thing. Based on the file sizes you show, this is a full model, not a PEFT adapter. Try using AutoConfig or AutoModel from transformers to load the config/model.

If this was originally trained as a PEFT model, ensure that only the PEFT adapter is saved, not the whole model. For instance, don't merge the adapter before saving the model. I can't tell what exactly happened here though, you need to investigate that.

BenjaminBossan avatar Apr 19 '24 08:04 BenjaminBossan

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

github-actions[bot] avatar May 18 '24 15:05 github-actions[bot]