Alex Cheema
Alex Cheema
What commit are you running on? Should be fixed with https://github.com/exo-explore/exo/commit/af734f1bf6cca5c13abf934391b2474093723e1b
> In my case happened the same and I solve it! > > After running exo, I copied the messages appearing in the terminal into Claude Sonnet to ask about...
> I have the same issue... I thought I had some how kicked off a 70B download and was trying to figure out how to cancel it 🤣 . Turns...
> Catch block that puts a little red `!` for the model status in the UI? > > If I get a chance I'll take a crack at a patch...
are you suggesting we use cashu for this?
Can you try on the latest commit with `DEBUG=2` and paste the entire output here please?
Downgraded to 1.70.0 https://github.com/exo-explore/exo/pull/800
> I test that tinygrad.Device.DEFAULT return value "GPU". When I delete Device.DEFAULT == "NV" in nvidia case exo worked.I don't know if it can work properly with oneAPI(Intel GPU) You'll...
Does this work as expected? If you look at mlx_parallm, they use a BatchedKVCache implementation to handle the kv cache for batches https://github.com/willccbb/mlx_parallm/blob/80b18ab49b80e6f8d82d89347ab32f44b35f8942/mlx_parallm/utils.py#L201 I'm not sure how it would work...
If someone could please convert/upload the weights then this would be supported out of the box. I've tried but my internet speed is too slow right now to do this....