LLaVA-NeXT
LLaVA-NeXT copied to clipboard
Add prompt format and sample inference code to HF model repos
Currently, the tokenizer_config is the same as the Llama 3 model, which isn't instructive as to how to pass in images.
Adding a very short snippet of code outlining how to load and inference the model would be a great addition. Same for the video repos.
Ideally inference could be done with either AutoModelForCausalLM or a LlavaLlama model (although I guess that has be created as the LLaVA NeXT Llama 3 model differs?)