Rangehow
Rangehow
在项目离线合成lora中有如下描述 > 合并Chinese-Alpaca-Plus需要提供两个LoRA权重,分别为Chinese-LLaMA-Plus-LoRA和Chinese-Alpaca-Plus-LoRA。执行以下命令完成合并: > > python scripts/merge_llama_with_chinese_lora.py \ > --base_model path_to_original_llama_hf_dir \ > --lora_model path_to_chinese_llama_plus_lora,path_to_chinese_alpaca_plus_lora \ > --output_type [pth|huggingface] \ > --output_dir path_to_output_dir > 参数选项含义与单LoRA权重合并中的含义相同。 需要注意的是 --lora_model参数后要提供两个lora_model的地址,用逗号分隔。 > > 两个LoRA模型的顺序很重要,不能颠倒。先写LLaMA-Plus-LoRA然后写Alpaca-Plus-LoRA。...
### System Info ```shell - optimum : 1.14.1 - huggingface_hub version: 0.19.4 - Platform: Linux-3.10.0-1160.88.1.el7.x86_64-x86_64-with-glibc2.17 - Python version: 3.9.17 - Running in iPython ?: No - Running in notebook ?:...
### Motivation I followed the guide [build-a-tokenizer-from-scratch](https://huggingface.co/docs/tokenizers/quicktour#build-a-tokenizer-from-scratch) and got a single tokenizer.json from my corpus. Since I'm not sure if it is compatible with the trainer, I want to convert...
### System Info - `transformers` version: 4.39.0.dev0 - Platform: Linux-3.10.0-1160.71.1.el7.x86_64-x86_64-with-glibc2.17 - Python version: 3.10.13 - Huggingface_hub version: 0.21.4 - Safetensors version: 0.4.2 - Accelerate version: 0.27.2 - Accelerate config: -...
I found here exist a difference in rope implementation mostly on permutation. Does this difference not affect the final result ? I'm not quite sure what I'm thinking. Sincerely ask...
I have tried unsloth+qlora, it's cool and brings considerable speedup and vram reduction. But I am not sure if this repo is useful for **full fine-tune** after searching repo/website/benchmark. So...
不完全的提取
https://www.neu.edu.cn/xygk/lrld.htm 从学校领导的提取只能提取到宁恩承
我在evaluation的issue里发现了很多和hf-mirror相关的问题 ``` File "/modules/evaluate_modules/metrics/evaluate-metric--exact_match/009c8b5313309ea5b135d526433d5ee76508ba1554cbe88310a30f85bb57ec88/exact_match.py", line 16 } ``` 这个问题是因为不完全下载导致的吗?
这个训练上比FA2更快,而且vram占用更少,在llama2的测试上非常有效。然而还不支持原生的qwen2,尽管有些三方脚本支持llamafy qwen,但是因为潜在的实现错误风险,让人不太有尝试欲望。unsloth已经被集成到llamafactory中。 benchmark:https://unsloth.ai/blog/mistral-benchmark#Benchmark%20tables 仓库:https://github.com/unslothai/unsloth
As title said, I made a minium snippet like this: ```python from transformers import ,Trainer class MyTrainer(Trainer): def compute_loss(inputs): input_ids = inputs.pop("input_ids") attention_mask = inputs.pop("attention_mask") valid_label_index_list = inputs.pop( "valid_label_index_list" )...