torchchat
torchchat copied to clipboard
[Llava][multimodal] enable Llava in torchchat
This PR aims to enable Llava in torchchat.
TODOs:
- [x] Create Model and ModelArgs as model definition entrances
- [x] Support model definition with multiple transformers
- [ ] Add transformer fusion types
- [ ] Enable Llava definition
- [ ] Add model config with checkpoint
- [ ] Update inference pipeline
:link: Helpful Links
:test_tube: See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/torchchat/966
- :page_facing_up: Preview Python docs built from this PR
Note: Links to docs will display an error until the docs builds have been completed.
:white_check_mark: No Failures
As of commit 11113ff322b63327f0d19f6ab84bfb9ce29c9119 with merge base 9dc9effd674463b572125d01760cb78752e1912a ():
:green_heart: Looks good so far! There are no failures yet. :green_heart:
This comment was automatically generated by Dr. CI and updates every 15 minutes.