Eric Curtin
Eric Curtin
@cooktheryan I want to open a PR that does something like this: ``` RUN if [ "$(uname -m)" = "aarch64" ]; then \ dnf copr enable -y slp/mesa-krunkit && \...
@smooge @gnaponie @lmilbaum are on this one
Building master branch now, I used to run something like this: ``` $ llama-main -m /models/granite --log-disable --instruct > Tell me about Georgi George Orwell (185-194) was an English writer...
--log-disable was added. I appreciate the advice, this is something from HuggingFace, I can't expect users to reconvert nor do I know how to. With --instruct this was not required,...
Like without some clean TUI interface for all models, like we had for "--instruct", I'm not sure what the use case is for "-cnv" ... Like if the use-case was...
I agree we should implement this. I would say even drop the "--format safetensors", I think it would not be too hard to automatically detect a certain model is safetensors.
Q4_0 or Q4_K_M are the middle of the road standard ones, but you might want to use other ones based on hardware, certain performance characteristics you want
From llamafile: When you download a new model with ollama, all its metadata will be stored in a manifest file under ~/.ollama/models/manifests/registry.ollama.ai/library/. The directory and manifest file name are the...
I think for simple usages of LLMs it's ideal... Like: ollama pull mistral is one of the reasons, ollama is so popular. It doesn't get easier than typing one word...
I noticed today, local-ai has it: https://github.com/mudler/LocalAI/pull/2628