Eric Buehler comments

Results 543 comments of


                                            Eric Buehler

Reduce CPU usage when idle

@scottwey thanks you for the testing! Great to see the 100% -> ~1% utilization :). If you could resolve the conflicts, we can absolutely merge.

0.3.1 #862 new build failure, stop at mistralrs-quant

@misureaudio that makes sense. I'll merge a nice solution!

0.3.1 #862 new build failure, stop at mistralrs-quant

@misureaudio @DenisBobrovskiy I just merged #878 which only compiles & runs the Marlin kernels if the compute cap is appropriate, can you please confirm if it works?

0.3.1 #862 new build failure, stop at mistralrs-quant

@misureaudio this was fixed in #944 .

The server crashed while deploying the gemma3 model.

@gqf2008 @chrootchad can you please share the command you used to reproduce this?

Build --features metal falis with "pattern `DType::F8E4M3` not covered"

@maslovw thanks for the issue! I opened #846 which should fix the build errors. Can you please run ``` git pull git switch metal_f8_bugfix ``` And then retry the build?

Build --features metal falis with "pattern `DType::F8E4M3` not covered"

@maslovw thank you for confirming! I merged #846, please feel free to reopen.

can't run gguf which download from ollama

@Sherlock-Holo I pushed [87a7c23](https://github.com/EricLBuehler/mistral.rs/commit/87a7c2368a61767a55e0f361b29c66792219c1c4) which fixes the loading. It looks like the chat template for these models in the GGUF file migth be incorrect, as it does not match the...

Metal Tensor failed assertion when shared between threads

Thanks for the issue and the example. I will take a look!

Handle assistant messages with 'tool_calls'

Friendly ping @Jeadie