Duc Quang Nguyen

icon indicating a website link https://cs.stanford.edu/~nqduc icon indicating an email [email protected]

icon company Ho Chi Minh City University of Technology icon location Viet Nam icon location CS Master Student

Results 3 comments of Duc Quang Nguyen

[Question] LLaVA Pretraining with Mixtral 8×7B

I have not faced this issue. Can you give me the reproducing command.

[Question] LLaVA Pretraining with Mixtral 8×7B

Have you merged my pull request about adding mixtral? If not, you can use my modified repo here: https://github.com/martinakaduc/LLaVA My pretraining script: `deepspeed llava/train/train_mem.py --deepspeed ./scripts/zero3_offload.json --model_name_or_path mistralai/Mixtral-8x7B-Instruct-v0.1 --version plain...

[Question] LLaVA Pretraining with Mixtral 8×7B

I think it is possible. However I have not tested yet.