Michael Han

Hi there thanks a lot for using us & so glad unsloth was useful! There was already an issue opened for Apple Silicon support: https://github.com/unslothai/unsloth/issues/4

I guess the main problem right now is Apple's ability to support Triton and other Unsloth reliance's. Making Triton work for Apple GPUs is a lot of work and Apple...

Yes it upcasts to 16bit for vllm so not an issue

Unsure sadly we've never tried it, but I'm assuming it works

Hey @mxjyst. Do you have a reproducible example for non unsloth? Have you tried our Colab notebooks to confirm? Also did you do benchmarking with unsloth first then hf in...

We're on it. Wil be slated for release later this year.

Yes thanks for being on the lookout! We will mostly likely be implementing this pretty soon!

Going to bump this again since it is a common request

@Sehyo do you know if the issue is now solved? Thanks