LLaVA-NeXT icon indicating copy to clipboard operation
LLaVA-NeXT copied to clipboard

Add prompt format and sample inference code to HF model repos

Open RonanKMcGovern opened this issue 1 year ago • 0 comments

Currently, the tokenizer_config is the same as the Llama 3 model, which isn't instructive as to how to pass in images.

Adding a very short snippet of code outlining how to load and inference the model would be a great addition. Same for the video repos.

Ideally inference could be done with either AutoModelForCausalLM or a LlavaLlama model (although I guess that has be created as the LLaVA NeXT Llama 3 model differs?)

RonanKMcGovern avatar May 16 '24 10:05 RonanKMcGovern