Unable to run Gradio demo: VILA with TinyChat

[mitraavi]

I've encountered a persistent issue while running the Gradio demo: "Gradio demo: VILA with TinyChat" on a local server, despite following the steps here: GitHub Link.

Problem: The model fails to load, and the Model dropdown menu in the Gradio UI doesn’t appear.

Details:

• Steps 1 and 3 (launching the controller and model worker) seem to work, with the controller returning a 200 response.
• Path to models "Efficient-Large-Model/VILA1.5-3b" (fp16-hf-model) and "vila-1.5-3b-w4-g128-awq.pt" provided in the model worker
• If I run Step-1,2,3, sequentially i noticed model passed in step-3 to controller never getting loaded in Gradio server. This means controller is unable to interact with Gradio server in the subsequent time as it is evident from screenshot here:

• if I run Step-1,3, first and step-2 at last then I can see that model in the "Model" section although unable to select it.

Overall I feel Controller is unable to interact with Graidio server after the 1st run. Any guidance on resolving this would be greatly appreciated.

[Lyken17]

@ys-2020 any ideas of what is happening?

[Lyken17]

We have a new demo interface which should be easier to use.

https://huggingface.co/spaces/Ligeng-Zhu/VILA

Feel free to reopen if any more issues.