Kevin Canwen Xu

This is for top-p sampling. We mask out tokens outside top-p (by default 95%).

Are you using LLaMA as the foundation model? If so, as LLaMA has no Chinese pretraining data, it's not very surprising that the outcome isn't very good. BLOOMZ may be...

Simply ```bash pip uninstall transformers pip install transformers ```

You can try install from source! Following this page: https://huggingface.co/docs/transformers/installation#installing-from-source or try `pip install git+https://github.com/huggingface/transformers.git`

> However a search of that Huggingface repository finds no mention of `LlamaForCausalLM` https://github.com/huggingface/transformers/blob/main/src/transformers/models/llama/modeling_llama.py

https://github.com/project-baize/baize/blob/ccf0bb8485657b7c16a57456bbb835503bac2456/demo/app.py#L18 Change this line to `tokenizer, model, device = load_tokenizer_and_model(base_model, adapter_model, load_8bit=True)`

Of course, but it won't be as good as a Baize model trained on a Chinese foundation model.

Not completely sure but this may be helpful: https://stackoverflow.com/questions/68652157/how-do-i-debug-overflowerror-value-too-large-to-convert-to-int32-t

Thanks for the pointer. This data may have copyright issue so we are very cautious about it and we can't at this moment provide checkpoints trained on it. But feel...

Hi @Keldos-Li Thanks for reaching out. Big fan of your project. Indeed I tried to fix the CSS for gradio on HF Space but it's not completely working. Thanks for...