grigio
grigio
I don't know, maybe a `--help` is enough and I don't know if it will be maintaned
Ok, if you make a PR I'll review it for merge
Here are some news.. but Linux support seems lacking.. https://community.amd.com/t5/ai/get-a-powerful-ai-assistant-with-document-chat-accelerated-by/ba-p/704092 https://lmstudio.ai/ryzenai
> Running a AMD AI 9 370HX here, Same as above. Hoping to see support Can you share how many token/s you get with llama3.1-Q4_k_m or similar ?
@JiapengLi I think Linux 6.14 should improve the situation, keep up updated https://www.phoronix.com/news/Ryzen-AI-NPU6-Linux-6.14
@GreyXor do you have improvements in tokens/sec over cpu or vulkan ?
+1 I use ollama in a docker container and I'd like to test it via HTTP api
I confirm the issue with `Mistral-Small-24B-Instruct-2501-Q4_K_M:24b`
I tried to run the smallest Flux.1 Schnell GGUF and I also have this issue ``` Error occurred when executing KSampler: HIP out of memory. Tried to allocate 72.00 MiB....
This is `rocminfo` ``` /opt/rocm/bin/rocminfo ROCk module is loaded ===================== HSA System Attributes ===================== Runtime Version: 1.14 Runtime Ext Version: 1.6 System Timestamp Freq.: 1000.000000MHz Sig. Max Wait Duration: 18446744073709551615...