llm
llm copied to clipboard
rustformers
Librarification
This is a very rough split of the CLI application into a library and application. It includes #8 because that was on my branch when I did this - we can figure out what to do with it when it's time.
The main differences are
- conversion of
printlns tologs - removal of
prints - adding a callback function for arrival of new tokens
Turns out this is basically entirely sufficient for first-pass use as a library. Here's proof.
Thanks for the feedback, everyone! Will action within the next few hours so we can get this merged 👍
The llama-rs crate now has no logging of its own; all progress is reported through the callback. Pretty happy with how that turned out!
I've also switched over to thiserror. I've tried to remain consistent with the existing names, but I'd like to agree on #17 before we have any major users.
As an aside: should the CLI be llama-cli or llama-rs-cli?
As an aside: should the CLI be llama-cli or llama-rs-cli
I was originally thinking llama-rs-cli. But after seeing the name, I really like llama-cli, especially if it ends up in someone's path via cargo install, it's easier to type. So if you prefer that one I'm on board as well!