stanford_alpaca
stanford_alpaca copied to clipboard
tatsu-lab
Code and documentation to train Stanford's Alpaca models, and generate the data.
When trying to load a model from output directory of train.py I get ``` Traceback (most recent call last): File "/usr/local/lib/python3.8/site-packages/transformers/modeling_utils.py", line 442, in load_state_dict return torch.load(checkpoint_file, map_location="cpu") File "/usr/local/lib64/python3.8/site-packages/torch/serialization.py",...
Hi, I have converted llama to HuggingFace format and now I'm trying to recover alpaca weights. However, after using the command `python weight_diff.py recover --path_raw --path_diff --path_tuned ` it prompts...
I am trying to re-train alpaca on the following machine:  ```bash +-----------------------------------------------------------------------------+ | NVIDIA-SMI 525.105.17 Driver Version: 525.105.17 CUDA Version: 12.0 | |-------------------------------+----------------------+----------------------+ | GPU Name Persistence-M| Bus-Id Disp.A...
Hey, I've been trying to use text-DaVinci-003 to generate self-instruct synthetic data from a complete domain-specific human-generated seed dataset, but the generated instructions are completely irrelevant to the domain, not...
I found the special tokens are not correctly set in tokenizer when i'm using `decapoda-research/llama-7b-hf`. Here is the code from [train.py](https://github.com/tatsu-lab/stanford_alpaca/blob/main/train.py) line[198](https://github.com/tatsu-lab/stanford_alpaca/blob/65512697dc67779a6e53c267488aba0ec4d7c02a/train.py#L198)-[206](https://github.com/tatsu-lab/stanford_alpaca/blob/65512697dc67779a6e53c267488aba0ec4d7c02a/train.py#L206) ``` special_tokens_dict = dict() if tokenizer.pad_token is None:...
After the LLaMA model finetuning using the existing training code, I realized that the model never outputs the EOS token, which causes the generation never stop until max_new_token is reached....
Hi, Could you consider changing the license to make it compatible with FLOSS distribution? Restriction of use, including commercial ones, are not compatible with FLOSS distributions, such as Debian and...
training log is ``` ng_rate': 5.263157894736842e-06, 'epoch': 0.02} {'loss': 1.2607, 'learning_rate': 6.31578947368421e-06, 'epoch': 0.03} {'loss': 1.2007, 'learning_rate': 7.368421052631579e-06, 'epoch': 0.03} {'loss': 1.1451, 'learning_rate': 8.421052631578948e-06, 'epoch': 0.04} {'loss': 1.1491, 'learning_rate': 9.473684210526315e-06,...
