Eric Buehler

@haricot were you planning on implementing HQQ for non-CUDA devices in this PR? The name seems to indicate so, I was just wondering!

@zackangelo @jorge-menjivar yes! I've been developing some things for this: - [x] "dummy" dtypes for mxfp4 in my Candle fork (since they are packed, we don't want to return unpacked...

> I definitely would be reluctant to suggest mistral.rs manage platform support on it's side when it's viable to delegate to a more suitable dependency like Candle / Burn? Switching...

> Did you manage to play with our rocm crate? @sonicrules1234 Not yet unfortunately, I've been working through some bugs and preparing for 0.6.0 (hoping to release very soon). Will...

> If there's any specific requirements mistral.rs needs that are blockers, perhaps those could be clarified? Depending on the state of their Candle backend, perhaps that'd ease a transition support...

> cubecl uses basically just hip bindings. I have wrappers for near all libraries. Now we introduced a struct ROCArray that represents a gpu array. it is a work in...

> nice, this is cool! I'll work on deleting from the prefix-cache if some byte-threshold was exceeded. Long term we could bring the trie structure back and do some vLLM...

So close: **master** (correct): ``` > hi Hello! How can I assist you today? > what is graphene Graphene is a two-dimensional material made of carbon atoms arranged in a...

Hi @TimDouglas2! I have opened an issue with `cudarc` and will let you know when it is resolved.

Hello @TimDouglas2! I just merged #424 which uses a new version of our CUDA backend driver which should support version 11.5. Can you please run ``` git pull cargo update...