Jan Kaniewski
Jan Kaniewski
There's a performance implication with recalculating the whole prompt each time (compute bound -> generating KV cache). That said supporting history storage and reply regeneration is absolutely an API that...
I think in the original API approach I was largely seeing `FLLMModelState` as a pure transform accessible on the game thread of the internal model state. Not sure if modifying...
This issue should be addressed in the https://github.com/getnamo/Llama-Unreal/releases/tag/v0.9.0 release.
pull latest, added missing categories (untested) here: https://github.com/getnamo/Llama-Unreal/commit/bb4e1a59f71e80698350becdab4183ffa7fc1e0c
If you're using this fork I recommend to start by using the plugin releases found here: https://github.com/getnamo/Llama-Unreal/releases/tag/v0.3.0 they contain the correct compiled dlls that should make it drag and drop...
If you're building from source (Oculus5.3), you may possibly need to recompile the project with the plugin in it so it generates new binaries to match your custom engine. The...
NB: if you're using the CUDA branch, you may be missing CUDA 12.2 runtimes. CPU-only should work though.
Does the windows CPU only build work for anyone?
It's a static build so it's possible it has a hidden dll dependency that works on my system. Need to do a dll build for CPU and cuda and try...
Doing a bit of a refactor before I make another build. Will test that one on other pcs to debug what's failing on startup.