Thomas Antony comments

Repositories
Issues
Comments

Results 24 comments of


                                            Thomas Antony

Ability for `./main` to keep the model in memory and pass it more text

@simonw I am working on a fork with some python bindings here: https://github.com/thomasantony/llama.cpp/tree/feature/pybind

Create json api service

I have a proof of concept working with an existing web UI here: https://github.com/oobabooga/text-generation-webui/pull/447 It is very unpolished, but getting somewhere.

Using non LoRA Alpaca model

You should theoretically be able to run the same convert and quantize scripts on that model and use them with llama.cpp.

Remove all the code

> You may thumb me down. But you can't undelete the code on my laptop. You are hereby in violation of Crablang(TM) Trademark Policy(R) by deleting the code on your...