dpublic comments

Results 5 comments of


                                            dpublic

this is making me depressed

I agree with 0xmulch, you have to use the tools that are given to you because we can't turn back time. I see this eventually becoming a great tool for...

feat: add support for flash_attn

I know this is late for this pull request, but can you add a new env var so people can force the use of flash_attn? This would be a way...

Some Ollama models apparently affected by llama.cpp BPE pretokenization issue

Will this llama.cpp merge https://github.com/ggerganov/llama.cpp/pull/6965, fix this issue? The llama.cpp commit link in ollama is dated 4/30 and https://github.com/ggerganov/llama.cpp/pull/6965 was merged to llama.cpp on 5/9. So, it doesn't look like...

Failure when building docker from source code Dockerfile

To build it, you need to build it with the script which uses podman (RHEL) flags (--load, --push) from what I tried: ./scripts/build_docker.sh

Llama 3 BPE tokenization needs improvement

The llama.cpp commit link in ollama is dated 4/30 and https://github.com/ggerganov/llama.cpp/pull/6965 was merged to llama.cpp on 5/9. So, it doesn't look like this merge was included with the last 0.1.37...