torchchat
torchchat copied to clipboard
[Llava][multimodal] enable Llava in torchchat
This PR aims to enable Llava in torchchat.
TODOs:
- [x] Create Model and ModelArgs as model definition entrances
- [x] Support model definition with multiple transformers
- [ ] Add transformer fusion types
- [ ] Enable Llava definition
- [ ] Add model config with checkpoint
- [ ] Update inference pipeline