MiniCPM-V
MiniCPM-V copied to clipboard
OpenBMB
Lora微调后要怎么加载测试呢？
怎么合并呢？哪里有示例或者demo吗
On Fri, 28 Jun 2024 at 7:53 PM, lyc728 @.***> wrote:
先对模型进行合并，就可以推理了
— Reply to this email directly, view it on GitHub https://github.com/OpenBMB/MiniCPM-V/issues/298#issuecomment-2196538074, or unsubscribe https://github.com/notifications/unsubscribe-auth/A26FH73GSP2KYPQ3CFOQXW3ZJUXBVAVCNFSM6AAAAABJ5H5MTSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCOJWGUZTQMBXGQ . You are receiving this because you commented.Message ID: @.***>
怎么合并呢？哪里有示例或者demo吗 … On Fri, 28 Jun 2024 at 7:53 PM, lyc728 @.> wrote: 先对模型进行合并，就可以推理了 — Reply to this email directly, view it on GitHub <#298 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/A26FH73GSP2KYPQ3CFOQXW3ZJUXBVAVCNFSM6AAAAABJ5H5MTSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCOJWGUZTQMBXGQ . You are receiving this because you commented.Message ID: @.>
这个链接里面有个合并模型https://www.chenxublog.com/2024/04/23/single-card-fine-tuning-minicpm-merging-lora-and-converting-to-gguf-for-llama-cpp.html
问的太对了哥，给我愁死了，这readme写的八成是写错了，那个msg应该是model
写成model会报错，就用msg就好了，不用管返回值。但是在运行的时候，显存占多少啊？我这里24G显存不够用，多显卡运行又报错
有没有运行全量微调后加载测试的，我使用官方提供的lora微调的加载测试的会报错
Lora finetune后，用官方代码调试，device='auto' 改成自己的device， 比如cpu or gpu（否则报错：‘NotImplementedError: Cannot copy out of meta tensor; no data!’），测试通过，结果正常。
有没有运行全量微调后加载测试的，我使用官方提供的lora微调的加载测试的会报错
Lora finetune后，用官方代码调试，device='auto' 改成自己的device， 比如cpu or gpu（否则报错：‘NotImplementedError: Cannot copy out of meta tensor; no data!’），测试通过，结果正常。
@MonkeyNi 您好，请问用官方代码加载lora，到了msg = model.load_state_dict(vpm_resampler_embedtokens_weight, strict=False)，之后具体怎么进行测试啊
有没有运行全量微调后加载测试的，我使用官方提供的lora微调的加载测试的会报错
Lora finetune后，用官方代码调试，device='auto' 改成自己的device， 比如cpu or gpu（否则报错：‘NotImplementedError: Cannot copy out of meta tensor; no data!’），测试通过，结果正常。
更新最新代码再试下
参考chat代码就行
from chat import MiniCPMVChat, img2base64 import torch import json
torch.manual_seed(0)
from peft import AutoPeftModelForCausalLM
model = AutoPeftModelForCausalLM.from_pretrained( # path to the output directory path_to_adapter, device_map="auto", trust_remote_code=True ).eval()
vpm_resampler_embedtokens_weight = torch.load(f"{path_to_adapter}/vpm_resampler_embedtokens.pt")
msg = model.load_state_dict(vpm_resampler_embedtokens_weight, strict=False)
im_64 = img2base64('./assets/airplane.jpeg')
msgs = [{"role": "user", "content": "Tell me the model of this aircraft."}]
inputs = {"image": im_64, "question": json.dumps(msgs)} answer = chat_model.chat(inputs) print(answer) 请问是这样吗
from peft import AutoPeftModelForCausalLM import torch from chat import MiniCPMVChat, img2base64 import json
path_to_adapter="xxxxxx/output_minicpmv2_lora/checkpoint-200" model = AutoPeftModelForCausalLM.from_pretrained( # path to the output directory path_to_adapter, device_map="cuda:0", trust_remote_code=True ).eval()
vpm_resampler_embedtokens_weight = torch.load(f"{path_to_adapter}/vpm_resampler_embedtokens.pt")
model.load_state_dict(vpm_resampler_embedtokens_weight, strict=False)
im_64 = img2base64('xxxxxxxxxx')
msgs = [{"role": "user", "content": "What is the car mileage?"}]
inputs = {"image": im_64, "question": json.dumps(msgs)} answer = model.chat(inputs) print(answer)
Traceback (most recent call last):
File "/home/matthew_kwok_tictag_io/minicpm/MiniCPM-V/predict.py", line 25, in
试试改成model.chat(mage=im_64, msgs=json.dumps(msgs),tokenizer=tokenizer)
