LongChat icon indicating copy to clipboard operation
LongChat copied to clipboard

Published 20 hours ago • verified publisher icon DachengLi1

Web GUI for longchat

Open VVNMA opened this issue 2 years ago • 3 comments
trafficstars

Hi, thanks for the great job! But I'd like to know is there a web GUI for this longchat model, it seems that the model worker in Vicuna does not support this model directly, there should be a condense ratio process.

VVNMA avatar Jul 04 '23 03:07 VVNMA

@VVNMA Thanks for trying out! LongChat is compatible in FastChat that we have implemented the monkey patch in the load_model function. The model worker is using load_model so it should work. Let me know if it does not work and we will take time to fix it!

DachengLi1 avatar Jul 04 '23 03:07 DachengLi1

@DachengLi1 Thanks. I saw the replace in the model adapter. But for the gradio server, I have to modify the input and max output token to fit the long chat.

VVNMA avatar Jul 05 '23 06:07 VVNMA

How can I see an option on UI to increase the context length from 1024 to 16K?

jalajthanaki avatar Jul 15 '23 08:07 jalajthanaki