Gpt4All-webui icon indicating copy to clipboard operation
Gpt4All-webui copied to clipboard

Published 20 hours ago • verified publisher icon ParisNeo

Inference time

Open eshaanagarwal opened this issue 1 year ago • 2 comments

Can you please let me know the inference time for the models ?

As far as i know, running gpt4all models on cpu is quite slow. i would appreciate your help in this

eshaanagarwal avatar Jun 06 '23 11:06 eshaanagarwal

It highly depends on: The binding used The model size The activation of cuda

ParisNeo avatar Jun 12 '23 05:06 ParisNeo

Can you point me towards one of the best GGML based model and binding ! I really need it for. Project and I am confused

eshaanagarwal avatar Jun 12 '23 06:06 eshaanagarwal