LLM Module planned?

Open Ohmagar opened this issue 2 years ago • 3 comments

Hey there,

have you planned to integrate either of the available LLM modules? something like GPT4All, pyllama, llama.py or llama-cop-python? It would be cool to be able to integrate even the 7B models into some small scripts. Or just in general muck about.

Sep 30 '23 05:09 Ohmagar

I'm going to think about it.

Sep 30 '23 09:09 holzschu

Lovely to hear! Thanks for the reply.

Oct 01 '23 06:10 Ohmagar

I just tested in on my mac and I underestimated how much performance it really needs. Not only did a prompt like „name the most common pets“ take about 6 minutes, other prompts were straight up garbage answers (like Name the planets in the solar system. Answer after 4 minutes „1.A“ 😂) Don’t bother with it - it’ll not be worth it Performance wise for the next few years IMO. ✌️

Oct 02 '23 04:10 Ohmagar