Hao Zhang comments

Results 174 comments of


                                            Hao Zhang

Python function instead of CLI

@iRanadheer : Could you describe your use case? If neither CLI nor web server works, what kind of interface do you expect? Recently @suquark adds a feature for openAI-like API,...

Python function instead of CLI

openAI-like APIs have been added. Now you can use python's request library to send post to the openai-like endpoint. Closing.

Occupied GPU memory keeps growing with more talks

I suppose you are using the web interface? This shouldn't happen when you clear the history. Could you provide more details?

Langchain Support

we'll investigate LangChain integration and update in the thread

finetune with lora error

We won't fix this in the short term. Because this is a complicated lack of support in FSDP. It is hard for us to fix FSDP. We suggest you use...

Fine-tuning Vicuna-7B with Local GPUs: RuntimeError: Expected is_sm80 || is_sm90 to be true, but got false

I'm closing this issue because this seems to be a flash attention issue. We'll soon migrate to use the xformer (https://github.com/facebookresearch/xformers) in place of flashattention, as our internal tests show...

Connection errored out on a AWS Sagemaker notebook

is websocket allowed on your sagemaker env?

NETWORK ERROR DUE TO HIGH TRAFFIC. PLEASE REGENERATE OR REFRESH THIS PAGE.

Are you serving your model to many users? If you have much traffic, the semaphore will allow only 4 users to use. The rest of the users have to wait...