stanford_alpaca
stanford_alpaca copied to clipboard
tatsu-lab
Code and documentation to train Stanford's Alpaca models, and generate the data.
我尝试过换成更小的模型，例如`https://huggingface.co/`中的`gpt2-small-chinese-cluecorpussmall'`，但是得到的错误码是相同的情况。 ``` (lmflow) [root@a4113ca43b08 LMFlow-main]# ./scripts/run_finetune.sh [2023-04-15 13:15:13,800] [WARNING] [runner.py:186:fetch_hostfile] Unable to find hostfile, will proceed with training with local resources only. [2023-04-15 13:15:13,815] [INFO] [runner.py:550:main] cmd = /home/minicoda3/envs/lmflow/bin/python -u...
Hi, I am trying to train the model `llama-7b-hf` with single GPU. I tried to reduce some parameters but I don't know if they are better. Components of my pc...
Hi, I got one question, I have seen the `evaluation_strategy` in the [training script](https://github.com/tatsu-lab/stanford_alpaca#fine-tuning). But I do not know the meaning for that parameter? Does that mean if I set...
Hi, I got one question that how to add validation dataset or record validation loss into the weight and bias during the [fine-tuning process](https://github.com/tatsu-lab/stanford_alpaca#fine-tuning)? Thanks in advance.
I use five V100-32G GPUs to train fine tune llama-7b and get OOM error every time. Here is the error messages: torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 388.00...
Hi, I got one question for the fine-tunining. I have seen the fine-tuning hyperparameters for training Alpaca-7B and Alpaca-13B. I feel confused about the batch size and epoch. from this...
1. The weight convert script of llama in https://github.com/huggingface/transformers/blob/main/src/transformers/models/llama/convert_llama_weights_to_hf.py#L101 seems have something wrong? There is a function called permute, which is called on each q_proj/k_proj weight ```python def permute(w): return...
Hi, do you mind share the code to plot the figure? I found the original notebook but was unable to reproduce it. thanks!
