LLaVA icon indicating copy to clipboard operation
LLaVA copied to clipboard

Training LLaVA MPT from scratch. Few Questions

Open chanangad opened this issue 9 months ago • 0 comments

Discussion

I noticed that the training scripts for MPT model have been removed and the latest code doesn't support pre-training and training with MPT base model. Was there a reason for removing the training scripts?

I also wanted to know that which conversation template was finally used for pre-training and instruction-tuning if anyone remembers. Was it the 'plain' template for pre-training and 'mpt' for instruction-tuning or something else?

Also was mm_use_im_start_end and mm_use_im_patch_token used in both the stages?

It'd be great if someone can help me with this,

chanangad avatar May 16 '24 20:05 chanangad