Eric Curtin

What I want to do here is get rid of the libexec files and just replace them with `ramalama` binary calls or move the files to bin rather than libexec....

@sarroutbi @ieaves got cycles for this one? I have to travel to Czech Republic this week

We had intended on merging vllm support soon, we started it here: https://github.com/containers/ramalama/pull/97 this is what we think an outline of what it should look like, basically we want to...

@rhatdan merged the first vllm-related PR, I dunno if you want to take a stab at implementing the other things you had in mind @p5

Best of luck @p5 @bmahabirbu did have success running on Windows recently: https://github.com/containers/ramalama/tree/main/docs/readme

> FYI - Ollama is now implementing vision models, so once v0.4 is released, it might be easier to integrate here. Indirectly maybe, we inherit from the same backend llama.cpp,...

And we wrote the Ollama transport from scratch, so we use zero Ollama code. What a lot of people don't realize is it's llama.cpp that does most of the heavy...

@vojtapolasek sure to point you in the right direction, we build our own version of the binaries here: https://github.com/containers/ramalama/blob/main/container-images/scripts/build_llama_and_whisper.sh and most things in ramalama result in a "podman run" command...

@ieaves I agree, I regard this closed since multi-modal got in

@rhatdan this is essential for the podman machine use case on macOS https://github.com/ericcurtin/ramalama/issues/14 Migrated from above, @cooktheryan has an idea how to do it... I don't know how to automate...