Update VisualQnA example with Falcon VLM

[arun-gupta]

trafficstars

Update VisualQnA example that uses Falcon VLM.

This would require to include Falcon as part of the validation at https://github.com/opea-project/GenAIComps/tree/main/comps/llms. And then create an updated VisualQnA that would use this microservice to use Falcon VLM.

[lucasmelogithub]

Supporting Falcon-11B would be great.

[kevinintel]

TGI-Gaudi did not support this model: https://huggingface.co/tiiuae/falcon-11B-vlm

we need to wait for TGI-Gaudi

[chickenrae]

@kevinintel can we move forward with this issue now?

[kevinintel]

we can't do anything unless tgi-gaudi supports it

[arun-gupta]

@kevinintel can this be done using a larger Xeon instance?

[lucasmelogithub]

@kevinintel can this be done using a larger Xeon instance?

VisualQnA works great on Xeon, I tested with a AWS "c7i.24xlarge" instance (96 vCPU 4th Gen Xeon w/ Intel AMX) using the llava-hf/llava-v1.6-mistral-7b-hf Model.

But when using tiiuae/falcon-11B-vlm, the "tgi-llava-xeon-server" microservice errors out.

[chickenrae]

@kevinintel What are the next steps to resolve this issue?

[xiguiw]

@arun-gupta @chickenrae

Thanks for create this topic. It's inactive for long time. Closet it now.

If anyone is interested in this topic, feel free to raise a new ticket.