Bradley Fox
Bradley Fox
Pretrain from scratch or modify the first ~1000 lines of the vocab.txt file with the vocab you'd like to add.
@peregilk Are you able to push that code up to a repo and link back here? It would be useful for many.
> @bradfox2 , @peregilk > You can use a modified version of Tensor2Tensor/text_encoder_build_subword.py code to generate BERT compatible vocab. > https://github.com/kwonmha/bert-vocab-builder That is also available in the BERT repo. The...
> > Pretrain from scratch or modify the first ~1000 lines of the vocab.txt file with the vocab you'd like to add. > > @bradfox2 What are we supposed to...
This was driving me insane. Sucks to see this as the issue.
@arrbhadri Can you more clearly explain your workaround?
Thanks for the response. More concerned about training in a bunch of newlines when using the provided tokenizer. Removing intermediate newlines from the output - or simply using the flan...
@DachengLi1 Thank you for the answer. I was not aware of that difference in standard vs Fast. Makes sense now.
GGML - not yet - https://github.com/ggerganov/llama.cpp/issues/247 GPTQ - not really - you can quantize but it is not very good - https://github.com/qwopqwop200/GPTQ-for-LLaMa/issues/157
@zhisbug AFAIK just T5