Amiber comments

Repositories
Issues
Comments

Results 2 comments of


                                            Amiber

infer中如何载finetuning的模型

按照这个代码： from transformers import AutoModel import torch # model = AutoModel.from_pretrained("THUDM/chatglm-6b", trust_remote_code=True, load_in_8bit=True, device_map='auto') model = AutoModel.from_pretrained("THUDM/chatglm-6b", trust_remote_code=True, device_map='auto') from transformers import AutoTokenizer tokenizer = AutoTokenizer.from_pretrained("THUDM/chatglm-6b", trust_remote_code=True) from peft import...

建议代码更新一下

在V100上完成predict代码如下： from transformers import AutoModel import torch model = AutoModel.from_pretrained("THUDM/chatglm-6b", trust_remote_code=True, device_map='auto') from transformers import AutoTokenizer tokenizer = AutoTokenizer.from_pretrained("THUDM/chatglm-6b", trust_remote_code=True) from peft import PeftModel model = PeftModel.from_pretrained(model, "./output/") import json...