Angad Chandhok issues

Repositories
Issues
Comments

Results 2 issues of


                                            Angad Chandhok

Pre-training with MPT-7B went well but fine-tuning it further gives garbled/random outputs

### Discussion After a few bug-fixes, I ran the pre-training code using mosaicml/mpt-7b model. The pre-training script I used ` deepspeed train_mem.py \ --deepspeed ./scripts/zero2.json \ --model_name_or_path mpt-7b \ --version...

Training LLaVA MPT from scratch. Few Questions

### Discussion I noticed that the training scripts for MPT model have been removed and the latest code doesn't support pre-training and training with MPT base model. Was there a...