[New Model]: Adding MiniGPT4_video model

[KerolosAtef]

The model to consider.

Github code :https://github.com/Vision-CAIR/MiniGPT4-video

huggingface demo : https://huggingface.co/spaces/Vision-CAIR/MiniGPT4-video

huggingface package : https://huggingface.co/Vision-CAIR/MiniGPT4-video-llama-hf example of using huggingface package

from transformers import AutoModel
video_path="path/to/the/video"
instruction="Write your question here"
use_subtitles=True
minigpt4_video_obj=AutoModel.from_pretrained("Vision-CAIR/MiniGPT4-video-llama-hf",trust_remote_code=True)
minigpt4_video_obj.to("cuda")
minigpt4_video_obj.eval()
answer=minigpt4_video_obj.inference_fun(video_path,instruction,use_subtitles)
print(answer)

The closest model vllm already supports.

meta-llama/Llama-2-70b-hf

What's your difficulty of supporting the model you want?

there are no new parts, but it is hard for me to integrate with vllm, I read the tutorial but can't do it. Your help in that will be greatly appreciated, as vllm is a very great framework for inference.

Also, this project is based on MiniGPT4 for images , which also will be great if integrated with vllm https://github.com/Vision-CAIR/MiniGPT-4