RDouglas comments

Results 7 comments of


                                            RDouglas

Finetuning Llama3-8B fails due to lack of a PAD token in new tokenizer

File "/home/doug/FastChat/fastchat/train/train.py", line 114, in preprocess raise ValueError( input_ids = tokenizer( ValueError: File "/home/doug/FastChat/venv/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 2858, in __call__ Asking to pad but the tokenizer does not have a padding token....

Finetuning Llama3-8B fails due to lack of a PAD token in new tokenizer

I think this is the issue: https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct/discussions/4

Finetuning Llama3-8B fails due to lack of a PAD token in new tokenizer

So here is my workaround while we wait for proper training support... First, implement the fixes suggested in the comments to issue: https://github.com/lm-sys/FastChat/issues/3263 This will add Llama-3 model adapter, and...

Finetuning Llama3-8B fails due to lack of a PAD token in new tokenizer

Only if you want output/training on any sample to end at any unknown token. Or that is my interpretation...

Finetuning Llama3-8B fails due to lack of a PAD token in new tokenizer

I ended up switching to Axolotl to train. I find it runs significantly faster and supports Llama3 properly. On Sun, Jul 7, 2024 at 6:00 AM Mellen Y. Pu ***@***.***>...

Is there an interactive mode?

Just clone the code from lit-parrot chat.py into lit-llama/generate.py ... that will give you an interactive mode.

Convert lit-llama weights to huggingface

Piling on here, comments in scripts/convert_hf_checkpoint.py say it's doing the inverse of https://github.com/huggingface/transformers/blob/main/src/transformers/models/llama/convert_llama_weights_to_hf.py , so it would be reasonable to assume that Immediately after creating a .pth model with convert_hf_checkpoint.py...