stanford_alpaca
stanford_alpaca copied to clipboard
tatsu-lab
Code and documentation to train Stanford's Alpaca models, and generate the data.
Thanks for this important work in pushing open LLMs forward. You mention, that you deviate from `self-instruct` in that you add an explicit (fixed) prompt for each and every instruction/input/output...
This repo is awesome .please let me know steps to use llama 13b to train similar json data like alpaca_data.json I have my custom data content and want to train....
Hi. I got some free quotas for Google Cloud TPU and I tried to run the training on it these two days. I did the following: 1. Create a TPU...
ChatGPT's continuous conversation capability amuses me a lot. So I wonder does Alpaca perform well on this?
Sorry to see the demo go dark. Hope you guys are doing ok. Wondering if you could run benchmarks with the weights you have against BIG-Bench Hard and share the...
I got this error ```bash python3.10/site-packages/transformers-4.27.0.dev0-py3.10.egg/transformers/trainer.py", line 1460, in _wrap_model self.model = model = FSDP( TypeError: FullyShardedDataParallel.__init__() got an unexpected keyword argument 'forward_prefetch' ``` [torch1.12](https://pytorch.org/docs/1.12/search.html?q=forward_prefetch&check_keywords=yes&area=default) does not support `forward_prefetch` ....
In the current [fine-tuning implementation](https://github.com/huggingface/transformers/blob/main/src/transformers/models/llama/tokenization_llama.py#L59), the eos token `` is not automatically added to the end of the input id. Therefore, the model is never trained to produce eos after...
