Alpaca-LoRA-Serve icon indicating copy to clipboard operation
Alpaca-LoRA-Serve copied to clipboard

Published 20 hours ago • verified publisher icon deep-diver

Internet search is very slow using orca mini on 4bit in google colab T4 on gradio

Open githubpradeep opened this issue 1 year ago • 3 comments

Screenshot 2023-07-26 at 1 59 47 PM

githubpradeep avatar Jul 26 '23 08:07 githubpradeep

i waited for more than 15 minutes and didnt get the response

githubpradeep avatar Jul 26 '23 08:07 githubpradeep

which orcamini did you choose? 7B or 13B?

deep-diver avatar Jul 26 '23 09:07 deep-diver

i chose 13B and loaded it in 4bit

githubpradeep avatar Jul 27 '23 09:07 githubpradeep