Lian Junhong
Lian Junhong
How to load a model pre-trained on a 52k dataset and continue fine-tuning with another dataset.json?
> I'm doing the same thing and the approach suggested by @chainyo in [#92 (comment)](https://github.com/tloen/alpaca-lora/issues/92#issuecomment-1479065352) worked for me. > > Here are the exact steps I'm using: > > *...
How to load a model pre-trained on a 52k dataset and continue fine-tuning with another dataset.json?
> My running version is more heavily customized, but here are the minimal needed changes: #154 I set ` CONTINUE_FROM_CHECKPOINT = "./alpaca-lora-7b"` which is download from huggingface and find ```...
How to load a model pre-trained on a 52k dataset and continue fine-tuning with another dataset.json?
> I tried the same to make sure. Got the weights from hf, continued training from them, no issue > > ``` > Restarting from ./lora-alpaca/alpaca-lora-7b/adapter_model.bin > trainable params: 4194304...
It seems your format is a bit confusing, which I can't understand very well.
同问，如何输出可训练参数量
> ### Is there an existing issue for this? > * [x] I have searched the existing issues > > ### Current Behavior > 看代码似乎默认是不开启prefix_projection，我在我的数据集上微调发现不开启prefix_projection的情况损失一直震荡不收敛，开启prefix_projection能收敛，但训练参数量比例从0.5%升到14%。这种情况正常吗？ > > ### Expected Behavior...
> > `preprocess_function_eval` 这个函数在调用 tokenizer 的时候已经设置了 `padding=True`，返回的输入已经是定长的了。目前还没有其他人反映过这个问题 > > 找到根本原因了，是因为我数据里target中存在null。 我也有一样的问题，我也检查一下训练数据，能否使用某些方法跳过null的数据
tokenizer = AutoTokenizer.from_pretrained( args.model_name_or_path, cache_dir=args.cache_dir, padding_side="right", **use_fast=True**, # Fast tokenizer giving issues. tokenizer_type='llama' if 'llama' in args.model_name_or_path else None, # Needed for HF name change )
系统设置，隐私与安全性，滑下去选仍然打开
> https://www.boatcloud.cn/d/8a9e362b-aa62-9995-f823-a2a09e1fcaa4 请问如何