refiners
refiners copied to clipboard
WIP new LLaVa implementation
Still working on the LLaVa architecture. It tooks longer than expected to understand the architecture and how the generate method works on a multimodal aspect. I'll go with the huggingface implementation that might be found under the following repo: llava-1.5-7b Remaining work:
- [x] create LLavaMeta architecture
- [x] add the mm_projection module
- [ ] implement the prepare_inputs_labels_for_multimodal
- [ ] implement the image processor method
- [ ] implement the LLaMa model (+ its tokeninzer)
- [ ] implement the weights conversion script
This bounty is stale because it has been opened for 7 days with no activity.
This bounty is stale because it has been opened for 7 days with no activity.
This bounty was closed because it has been inactive for 7 days since being marked as stale.
This bounty is stale because it has been opened for 7 days with no activity.
This bounty is stale because it has been opened for 7 days with no activity.
This bounty is stale because it has been opened for 7 days with no activity.
This bounty is stale because it has been opened for 7 days with no activity.
This bounty is stale because it has been opened for 7 days with no activity.
This bounty is stale because it has been opened for 7 days with no activity.
This bounty is stale because it has been opened for 7 days with no activity.
This bounty is stale because it has been opened for 7 days with no activity.
This bounty was closed because it has been inactive for 7 days since being marked as stale.