mahmoodsh36
mahmoodsh36
just as a small follow-up, current work (not much) is present at https://github.com/mahmoodsh36/organ-mode https://github.com/mahmoodsh36/cltpt
hello, afaict kv cache quantization is not yet available in mistral-rs? i have been using it with llama-server (llama-cpp) because it allows me to use a considerably longer context size...
for context, i am on running from flake on nixos, im using hyprland (wayland). the issue can be simply seen by looking at the right portion of the screen, a...
i would love to see this too.
im receiving similar errors