unsloth Triton 3.0.0 does not work

Hello,

I've been playing around with the collab examples on both AMD and NVDA sides and noticed that when using Triton's latest (main branch) it seems to break down at Unsloth's init step in the init.py:

https://github.com/unslothai/unsloth/blob/27fa021a7bb959a53667dd4e7cdb9598c207aa0d/unsloth/init.py#L70C33-L70C45

I noticed that Triton's latest package structure changed a little compared to the old releases, the /common package (which is assumed in Unsloth's src code) no longer exists. I dug a little more and found the last version that had the old package structure was around the 2.1.0 release... (and any other versions around this one).

New Triton structure:

https://github.com/triton-lang/triton/tree/main/python/triton

Old Triton structure:

https://github.com/triton-lang/triton/tree/v2.1.0/python/triton

I was wondering, if there are any plans to support latest version of Triton moving forward?

Jun 03 '24 20:06 jhu960213

Oh someone did report this a while back, but I forgot about it whoops - will relook into this!

Jun 04 '24 15:06 danielhanchen

Thanks! I'll check back later on

Jun 04 '24 19:06 jhu960213

I think I fixed it!

Jun 16 '24 16:06 danielhanchen

Nice! I'll test it out later this week :)

Jun 17 '24 16:06 jhu960213

Hi @danielhanchen, sorry for the delay, just was able to test it out now, it seems like with triton 3.0.0 (installing from src from their main branch) BitsandBytes is failing now.... did you guys encounter this issue as well?

Jul 17 '24 21:07 jhu960213

Oh that seems like bitsandbytes themselves have not updated their package to work with Triton 3

Jul 18 '24 07:07 danielhanchen

Seems like it yeah. What dependency versions are you using to test triton 3.0.0 in Unsloth? @danielhanchen

Jul 18 '24 20:07 jhu960213

Oh I used torch 2.5 and the latest xformers - tbh I haven't tested bitsandbytes

Jul 20 '24 20:07 danielhanchen

I see..... so bitsandbytes works with upstream triton (from cached pypi server) but I'm seeing an error in xformers (0.0.26.post1) in one of their kernels when they try to jit()

Jul 24 '24 16:07 jhu960213

Hmmm I'll try investigate this

Jul 26 '24 06:07 danielhanchen

Were you able to see a jit() failure in the 0.0.26 version of xformers? What about in the newest release of xformers?

Jul 29 '24 15:07 jhu960213

The latest version of xformers should work (BUTT Unsloth needs < 0.0.27 lol) - I first need to fix that first!

Jul 31 '24 02:07 danielhanchen

Haha, yeah I saw that in the src code, did you get a chance to change the version requirements?

Aug 06 '24 15:08 jhu960213

Working on it!

Aug 10 '24 03:08 danielhanchen

Changed the requirements! Xformers > 0.0.27, Torch 2.4/2.5 and Python 3.12 all work now!

Aug 12 '24 00:08 danielhanchen

Great! I will try it out soon later this week.

Aug 12 '24 18:08 jhu960213

so unsloth can works with amd now?

Oct 05 '24 14:10 lastrei

@lastrei Maybe - we're tracking AMD support in https://github.com/unslothai/unsloth/issues/37

Oct 09 '24 06:10 danielhanchen

I'll close this issue for now! Triton 3 definitely functions with Torch 2.4 :)

Oct 09 '24 06:10 danielhanchen

Hey, sorry I've been a bit busy lately, closing the ticket is fine. I haven't found time to test it yet.... I'll circle back if there are any issues in the future :)

Oct 10 '24 21:10 jhu960213