rustformers
how should i build a web interface?
Is there anyway to provide a history of prompts (like chat-gpt4) and tell the llm-cli to return json?
Your best bet is to build your own server based around llm; llm-cli is basically just a demo application for llm.
You can see how this might be done in the now-closed PR #37.
it looks like that PR was closed and llm-http isn't actually a thing yet. I'm not good with rust, but I'm good with node. I can call out to llm-cli but was just curious if there's another way.
Do you know how I could track a history of prompts with llm-cli?
Have you tried using https://github.com/Atome-FE/llama-node? I think it has bindings to this library
This is a nodejs library for inferencing llama, rwkv or llama derived models. It was built on top of llama-rs, llama.cpp and rwkv.cpp. It uses napi-rs for channel messages between node.js and llama thread.
(this repo used to be called llama-rs until recently)
@ralyodio I'm going to close this Issue, but please feel free to reopen it or open a new one if you have more questions.
