Zhihong Chen comments

Results 31 comments of


                                            Zhihong Chen

Would you provide some details of model training process?

Hi @intelligencegear and @RahulBhalley, Thanks for your attention! We have uploaded the training code (see [here](https://github.com/FreedomIntelligence/LLMZoo#-training-by-yourself)). Now you can train `Phoenix`. :-) We specify all the hyper-parameters in the shell...

running model with 8-bit, too slow!

Hi @ananwjq, Thanks for your feedback! We are working on efficient inference and will give a detailed comparison. Best, Zhihong

About the tokenizer

Hi @yuyq96, Thanks for your attention! Could you upgrade the `transformers` version: `pip install git+https://github.com/huggingface/transformers` and try again? Best, Zhihong

out-of-bounds problem with the targets array in med-vqa2019

Hi all, Please check if the dataset is the same one used in the codebase. Here is the [link](https://drive.google.com/file/d/1QU7OFll-vpB7UNLySoyUJbxVJfc_Y9zS/view?usp=sharing). Best, Zhihong

对安装环境有要求吗？

Hi @feiyacz, 可以提供一下更具体的信息吗，比如其中哪个库安装失败。 Best, Zhihong

Cannot resume trainer from checkpoint

Hi @YerayL, Could you provide the `transforemers` version? Best, Zhihong

How to fix this issues "Failed to build flash-attn ERROR: Could not build wheels for flash-attn, which is required to install pyproject.toml-based projects"

Dear @LiZhangMing, Thanks! For the issue about `Flast-Attn`, please refer to [this repo](https://github.com/HazyResearch/flash-attention). Alternatively, we can use `train.py` rather than `train_fast.py` to turn off the flash attention. Best, Zhihong

Zhihong Chen