An Xiao comments

Results 6 comments of


                                            An Xiao

[Usage] Some weights of LlavaLlamaForCausalLM were not initialized from the model checkpoint When I try to pretrain model.

> For the miss weights, I think it may caused by the transformer pkg version. I update it from 4.31.0 to 4.33.2 and solved. @ZizhenWang I faced the problem like...

[Question] LLaVA Pretraining with Mixtral 8×7B

@martinakaduc Thank you very much for your prompt reply! Below is my pretraining script. The --model_name_or_path is the model I downloaded from HF mistralai/Mixtral-8x7B-v0.1. Despite the warnings, running this script...

[Question] LLaVA Pretraining with Mixtral 8×7B

@martinakaduc Thank you! I will try it now and find out the problem!

[Question] LLaVA Pretraining with Mixtral 8×7B

@martinakaduc Hi, I'm using your pretrained MixSUraV model downloaded from HF to finetune on my own dataset. The script I use is like Figure 1, is it correct? If correct,...

[Question] LLaVA Pretraining with Mixtral 8×7B

> Hi, how do you know the training was effecitve? Did you use the default training setting? I LoRA with default parameters and basically no improvement. @fisher75 I have LoRA...

[Question] LLaVA Pretraining with Mixtral 8×7B

@fisher75 Sure, e-mail me your WeChat ID is ok.