sshu2017

I know this is closed but I still encountered this issue when loading ggml-v3-13b-hermes-q5_1.bin - tried gpt4all==0.3.4 and gpt4all==0.3.0 but both gave me this error...

So, I've tested the n_threads on AWS EC2, and so far the optimal value is 48. I don't understand why but with 72CPUs and 96CPUs, the response speed slowed down...

> Hi @abhishekrai43 , sorry for the late reply. With more threads, now I can get a response in ~30 seconds. It was ~150 seconds with everything to the default...