Eric Buehler
Eric Buehler
Glad that it works!
Hi @FL33TW00D! I was wondering what your thoughts were on being able to integrate Ratchet as a backend - do you think the API is complete enough?
Ok. Do you have a rough timeline on that, and do you anticipate any major API changes? I may start working on a backed agnostic framework for mistral.rs like Burn...
> Should be released next week. There may be API changes for the foreseeable future, it's not 1.0 yet. Exciting! > Burn may be a good backend! No quant support...
> We will support the most popular GGUF quants, and then expand to better quant schemes. I'm adding GPTQ here: #467. > Quantization is completely transparent, no QMatmuls here. How...
Hi @ilovethensa! ROCm support is definitely on the radar, however it will be in one month at earliest because of the other projects and exciting features which are upcoming. OpenCL...
@grinapo @Sherlock-Holo I merged some fixes, can you please try this again after `git pull` and rebuild?
@Sherlock-Holo @grinapo this error should be somewhat fixed in https://github.com/EricLBuehler/mistral.rs/issues/1137#issuecomment-2657877915. I would recommend specifying a chat template explicitly, as documented [here](docs/CHAT_TOK.md).
@sopaco I cannot reproduce this, can you please run `cargo update` to ensure you are using the latest versions? If that fails, can you please run `cargo tree`?
> I'm really struggling to understand if this cacher is currently being used. It is being used by default. If you set the prefix cache n to be 0, then...