Eric Curtin
Eric Curtin
@mattnewsome I just made updates to the installer can you try again?
@mattnewsome yeah our containers solution is in a funny place, you need to manually configure podman-machine to use krunkit, then it should work. podman-machine doesn't have /dev/dri with the default...
I brought this up before, but I believe we should default to what Docker is doing: https://www.docker.com/blog/why-docker-chose-oci-artifacts-for-ai-model-packaging/ it's fine. It could be easily extended to multi-modal (and vllm). Multi-modal gguf...
Try "--runtime-args" "--jinja"
I'm gonna open a PR to enable --jinja everywhere, see if anything starts to fail... If things start to fail, I really thinks it's an issue that needs to be...
This is the wrong space to discuss these things in general, this is more on the llama.cpp side to address.
I agree we can close this @engelmi llama.cpp now has tool calling with jinja, I don't think it's 100% complete but a lot of people are using the bits available...
The solution is: https://github.com/docker/model-runner
> I am afraid the changes still add to much extra logic that we will have to work around over time and it does not justify the small added benefit....