无法结合原始重量和增重

[kikotrtr1]

在执行 python -m fastchat.model.apply_delta --base llama-13b/ --target save/working/vicuna/weight/ --delta vicuna-13b-delta-v0/ 命令的时候

顺利读取了原始文件和增重文件

然后报错了，有大神能看看是什么导致了，检查了以下，库应该是完整安装了的

[gch8295322]

Change "tokenizer_class": "LLaMATokenizer" in llama-13b-hf/tokenizer_config.json into "tokenizer_class": "LlamaTokenizer". It worked for me~

[huangzhongzhong]

I didn't encounter this problem alone

[kikotrtr1]

感谢～

发自我的iPhone

------------------ 原始邮件 ------------------ 发件人: gch8295322 @.> 发送时间: 2023年4月20日 01:45 收件人: Vision-CAIR/MiniGPT-4 @.> 抄送: kikotrtr1 @.>, Author @.> 主题: 回复：[Vision-CAIR/MiniGPT-4] 无法结合原始重量和增重 (Issue #59)

Change "tokenizer_class": "LLaMATokenizer" in llama-13b-hf/tokenizer_config.json into "tokenizer_class": "LlamaTokenizer". It worked for me~

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>

[Arnold1]

what os are you using? does it also work on osx using m1?

[Wenbobobo]

Change "tokenizer_class": "LLaMATokenizer" in llama-13b-hf/tokenizer_config.json into "tokenizer_class": "LlamaTokenizer". It worked for me

This issue is still here😥 （Windows terminal）

[kenneth104]

都是用git clone拉取的文件，可以成功合并。。就差一个显卡了

[TsuTikgiau]

@Wenbobobo Not sure if this comes from the version of HuggingFace. What is the HuggingFace version you are using?

[TsuTikgiau]

Hello @kenneth104 We now provide a model aligned with Vicuna7b. You can find it and the way to set it in the updated Readme. It can be launched in GPU with 12G memory if you keep the beam search width to 1.

[kenneth104]

@TsuTikgiau Thanks

But my local GPU only have 8G vRAM So ,waiting for CPU support,server can accept about 200G of memory and provide low inference rates

[Wenbobobo]

@Wenbobobo Not sure if this comes from the version of HuggingFace. What is the HuggingFace version you are using?

I download it from ( https://huggingface.co/decapoda-research/llama-7b-hf/tree/main ). I have been stuck in this question for a long time, and I am really looking forward to your answer. Thanks!(ง •_•)ง @TsuTikgiau

[kenneth104]

@Wenbobobo

Try to use this(Recommend use git lfs clone all the file),I'm use this can output the model https://huggingface.co/huggyllama/llama-13b/tree/main

[Wenbobobo]

@Wenbobobo

Try to use this(Recommend use git lfs clone all the file),I'm use this can output the model https://huggingface.co/huggyllama/llama-13b/tree/main

Do I need to convert it to hf format? Or should I directly combine it with vicuna weight? @kenneth104 Really thanks for your answer~

[ntdat017]

Change "tokenizer_class": "LLaMATokenizer" in llama-13b-hf/tokenizer_config.json into "tokenizer_class": "LlamaTokenizer". It worked for me~

It worked for me too...

[kenneth104]

@Wenbobobo Try to use this(Recommend use git lfs clone all the file),I'm use this can output the model https://huggingface.co/huggyllama/llama-13b/tree/main

Do I need to convert it to hf format? Or should I directly combine it with vicuna weight? @kenneth104 Really thanks for your answer~

directly use the "fastchat.model.apply_delta" to combine "vicuna" and "delta"

[LLLYF]

尝试使用这个（建议使用git lfs克隆所有文件），我用这个可以输出模型 https://huggingface.co/huggyllama/llama-13b/tree/main

我需要将其转换为 hf 格式吗？还是我应该直接将其与骆马重量结合使用？真的谢谢你的回答~

直接使用“fastchat.model.apply_delta”组合“骆马”和“三角洲”

很抱歉打扰您，想向您请教一下您的模型组合流程是否是通过git lfs直接克隆您所提供地址的内容，完成之后就接着执行组合代码呢？这样就无需再去找原始权重了是吗？

[WeileiZeng]

@Wenbobobo Try to use this(Recommend use git lfs clone all the file),I'm use this can output the model https://huggingface.co/huggyllama/llama-13b/tree/main

Do I need to convert it to hf format? Or should I directly combine it with vicuna weight? @kenneth104 Really thanks for your answer~

I think you need to convert it into HF. The script is in the transformer module

[LLLYF]

尝试使用这个（建议使用git lfs克隆所有文件），我用这个可以输出模型 https://huggingface.co/huggyllama/llama-13b/tree/main

我需要将其转换为 hf 格式吗？还是我应该直接将其与骆马重量结合使用？真的谢谢你的回答~

直接使用“fastchat.model.apply_delta”组合“骆马”和“三角洲”

很抱歉打扰您，想向您请教一下您的模型组合流程是否是通过git lfs直接克隆您所提供地址的内容，完成之后就接着执行组合代码呢？这样就无需再去找原始权重了是吗？

I seem to have found a solution. The key is to download the original weights and convert them into hf format, which is the weight file at the end of the bin file (we have already downloaded this bin file, so we need to find the original file without executing the conversion format command). Now, all we need to do is use the final command to combine the converted and original files with code.

[aries-ai-dev]

Change "tokenizer_class": "LLaMATokenizer" in llama-13b-hf/tokenizer_config.json into "tokenizer_class": "LlamaTokenizer". It worked for me~

thanks, it worked for me too. ^_^