LLaVA-NeXT

LLaVA-NeXT copied to clipboard

Demo deployment issues

4

Hi there! I clone the tensors here: git clone https://huggingface.co/lmms-lab/llava-next-interleave-7b And I do the setup as is in the readme (which needs an upgrade for gradio (pip install --upgrade gradio)...

pedrocolon93

+ Add assert to make sure number of images == number of image tokens in inputs + Fix the case [where num_images == 0](https://github.com/LLaVA-VL/LLaVA-NeXT/blob/inference/llava/model/llava_arch.py#L263): + We don't need to use...

khaimt

When I was trying to run the gradio demo with the new llava-next-interleave-7b model, I ran into the following two errors. Any ideas?

justinphen

When will the training code open sourced?

9

Thanks for your great job. When will the training code open sourced?

jimchenhub

Strange Model Loading Issue: Inconsistency with Vision Tower Parameters

2

I found something strange when loading the model. It seems that you have released the vision_tower during training, but when loading the vision_tower, you did not load the gradient-updating parameters,...

shidingz

Will the interleaved data opensource?

7

Will the interleaved data opensource?

lucasjinreal

Will LLaVA-NeXT-interleave 0.5B release?

1

I think the 0.5B model already shown strong performance in Multi-image Evaluation, will this model release later? Thanks.

zihaolucky

Issue running llava/eval/model_video_chatgpt_general.py - Missing eval folder

1

hi, I am trying to run `llava/eval/model_video_chatgpt_general.py`, but I noticed that there is no `eval` folder within the `llava` directory. Is that wrong?

sunwhw

LLaVa-NeXT-Video is added to 🤗 Transformers!

28

Hey all! The video models are all supported in Transformers now and will be part of the v4.42 release. Feel free to check out the model checkpoints [here](https://huggingface.co/collections/llava-hf/llava-next-video-6666a9173a64c7052930f153). To get...

zucchini-nlp

[Question] Is this slated for release in the Transformers library?

8

Is this going to be on the transformers library? Seems like it's going to be big.

WAS-PlaiLabs